2024-04-26T21:39:05.331 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:39:05.332 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} --owner scheduled_rishabh@teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/orig.config.yaml 2024-04-26T21:39:05.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:39:05.484 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7675232' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF8ro+zAIG4+52rmF8R1dZVNFuvV25HlLKAXxNpq77B8vaqJJEdgfs4FinReJF2lDqGCO5BALQW/8EX1hEdTZrk= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK19vZwYGt/Mo3nYrd8RnzosczujCaPFzfKXB+DM9tLUfoDFcYpYYugQHPp+FU43Z+pIYiIQKru8D2py8b4+8LM= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAteR6OsBnihUHls67V0BtT3HQ1xsQ3LoNEDom/tfdnP3+hw7901uD5EOzJTcimytI6kzslVNH8+7DDUGcEecDY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:39:05.484 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:39:05.485 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:39:05.485 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:39:05.486 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:39:05.511 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:39:05.511 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:39:05.511 INFO:teuthology.packaging:ref: None 2024-04-26T21:39:05.511 INFO:teuthology.packaging:tag: None 2024-04-26T21:39:05.511 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:39:05.512 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:39:05.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:39:05.653 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:39:05.655 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:39:05.661 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:39:05.661 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:39:05.678 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:39:05.700 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:39:05.708 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:39:05.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:33:56.356610', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF8ro+zAIG4+52rmF8R1dZVNFuvV25HlLKAXxNpq77B8vaqJJEdgfs4FinReJF2lDqGCO5BALQW/8EX1hEdTZrk='} 2024-04-26T21:39:05.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:33:56.358491', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK19vZwYGt/Mo3nYrd8RnzosczujCaPFzfKXB+DM9tLUfoDFcYpYYugQHPp+FU43Z+pIYiIQKru8D2py8b4+8LM='} 2024-04-26T21:39:05.763 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:33:56.360164', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAteR6OsBnihUHls67V0BtT3HQ1xsQ3LoNEDom/tfdnP3+hw7901uD5EOzJTcimytI6kzslVNH8+7DDUGcEecDY='} 2024-04-26T21:39:05.764 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:39:05.770 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T21:39:05.770 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T21:39:05.771 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T21:39:05.771 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:39:05.912 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f21e5606160>, signals=[15]) 2024-04-26T21:39:05.913 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:39:05.919 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:39:05.920 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T21:39:05.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:39:06.012 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T21:39:06.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:39:06.088 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-04-26T21:39:06.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:39:06.164 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:39:06.170 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-04-26T21:39:06.189 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-04-26T21:39:06.189 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-04-26T21:39:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-04-26T21:39:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-04-26T21:39:06.247 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:39:06.249 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-04-26T21:39:06.268 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-04-26T21:39:06.286 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-04-26T21:39:06.286 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-04-26T21:39:06.343 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-04-26T21:39:06.343 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-04-26T21:39:06.343 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-04-26T21:39:06.343 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:39:06.344 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:39:06.345 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-04-26T21:39:06.368 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-04-26T21:39:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-04-26T21:39:06.387 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-04-26T21:39:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:39:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:39:06.446 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-04-26T21:39:06.468 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:39:06.478 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:39:06.484 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:39:06.484 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:39:06.487 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:39:06.489 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:39:06.505 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:39:06.511 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:39:06.511 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:39:06.544 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:39:06.546 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:39:06.671 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:39:06.748 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:39:06.763 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:39:06.763 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T21:39:06.763 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T21:39:06.763 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:39:06.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:39:06.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T21:39:07.020 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:39:07.020 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:39:07.020 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-26T21:39:07.038 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:39:07.039 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:39:07.039 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:39:07.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum upgrade -y linux-firmware 2024-04-26T21:39:10.262 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-04-26T21:39:13.321 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 19 MB 00:01 2024-04-26T21:39:17.522 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T21:39:19.147 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-26T21:39:20.784 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T21:39:26.023 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T21:39:28.036 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-26T21:39:28.037 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-04-26T21:39:28.037 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-04-26T21:39:28.150 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T21:39:28.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:39:28.294 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:39:28.294 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:39:28.296 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi012.front.sepia.ceph.com to complete... 2024-04-26T21:39:40.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26 21:39:40 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:39:40.408 INFO:teuthology.orchestra.run.smithi012.stderr:FINISHED --2024-04-26 21:39:40-- 2024-04-26T21:39:40.408 INFO:teuthology.orchestra.run.smithi012.stderr:Total wall clock time: 12s 2024-04-26T21:39:40.408 INFO:teuthology.orchestra.run.smithi012.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-26T21:39:40.412 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T21:39:40.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:39:42.653 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying... ######################################## 2024-04-26T21:39:42.797 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing... ######################################## 2024-04-26T21:39:42.925 INFO:teuthology.orchestra.run.smithi012.stdout:Updating / installing... 2024-04-26T21:39:51.284 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:40:31.625 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:40:31.625 DEBUG:teuthology.orchestra.run.smithi012:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:40:31.715 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:40:31.716 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:40:31.717 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:40:31.718 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:40:31.719 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:40:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:40:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:40:31.722 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:40:31.723 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:40:31.724 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:40:31.725 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:40:31.726 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:40:31.727 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:40:31.728 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:40:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:40:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:40:31.731 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:40:31.732 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:40:31.733 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:40:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:40:31.735 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:40:31.737 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:40:31.738 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:40:31.739 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:40:31.740 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:40:31.741 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:40:31.742 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:40:31.743 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:40:31.744 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:40:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:40:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:40:31.747 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:40:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:40:31.749 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:40:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:40:31.751 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:40:31.752 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:40:31.753 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:40:31.754 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:40:31.755 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:40:31.756 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:40:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:40:31.758 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:40:31.759 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:40:31.760 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:40:31.761 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:40:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:40:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:40:31.764 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:40:31.765 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:40:31.766 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:40:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:40:31.768 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:40:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:40:31.770 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:40:31.771 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:40:31.772 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:40:31.773 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:40:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:40:31.775 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:40:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:40:31.777 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:40:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:40:31.779 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:40:31.780 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:40:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:40:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:40:31.783 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:40:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:40:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:40:31.786 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:40:31.787 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:40:31.789 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:40:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:40:31.791 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:40:31.792 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:40:31.793 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:40:31.795 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:40:31.796 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:40:31.797 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:40:31.798 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:40:31.799 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:40:31.800 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:40:31.801 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:40:31.802 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:40:31.803 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:40:31.804 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:40:31.806 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:40:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:40:31.809 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:40:31.810 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:40:31.811 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:40:31.812 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:40:31.813 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:40:31.814 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:40:31.815 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:40:31.816 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:40:31.819 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:40:31.820 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:40:31.821 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:40:31.822 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:40:31.823 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:40:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:40:31.825 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:40:31.826 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:40:31.827 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:40:31.828 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:40:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:40:31.830 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:40:31.831 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:40:31.832 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:40:31.833 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:40:31.834 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:40:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:40:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:40:31.837 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:40:31.838 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:40:31.839 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:40:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:40:31.841 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:40:31.842 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:40:31.843 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:40:31.844 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:40:31.845 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:40:31.846 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:40:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:40:31.848 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:40:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:40:31.853 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.853 DEBUG:teuthology.orchestra.run.smithi012:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:40:31.877 INFO:teuthology.orchestra.run.smithi012.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: fi 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:40:31.878 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-04-26T21:40:31.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-04-26T21:40:31.967 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 68.el9 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8264254 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:40:31.968 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:40:31.969 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:40:31.970 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:40:31.970 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:31.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:40:32.490 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-04-26T21:40:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:40:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-04-26T21:40:33.847 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-26T21:40:33.862 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.khbO0loQPd 2024-04-26T21:40:33.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.khbO0loQPd 2024-04-26T21:40:33.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.khbO0loQPd 2024-04-26T21:40:34.069 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.khbO0loQPd is 6KB 2024-04-26T21:40:34.118 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.khbO0loQPd 2024-04-26T21:40:34.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:40:34.199 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:40:34.199 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:40:34.199 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:40:34.199 INFO:teuthology.orchestra.run.smithi012.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:40:34.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:40:34.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-04-26T21:41:04.377 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T21:41:04.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:41:04.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:41:38.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-26T21:41:47.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:41:47.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:41:51.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-26T21:42:03.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:42:03.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:42:03.432 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-26T21:42:04.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:42:04.107 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:42:04.107 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-26T21:42:04.165 INFO:teuthology.orchestra.run.smithi012.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:42:04.165 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:42:04.165 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:42:04.165 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:42:05.166 DEBUG:teuthology.parallel:result is None 2024-04-26T21:42:05.167 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:42:05.174 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:42:05.174 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:42:05.180 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:42:05.182 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:42:05.219 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:42:05.226 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:42:05.234 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:42:05.234 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:42:05.264 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:42:05.267 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:42:05.377 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:42:05.384 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:42:05.384 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:42:05.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:05.400 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:42:05.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:05.417 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:42:05.423 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:42:05.443 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:42:05.510 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.517 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.529 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.533 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.542 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.584 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:42:05.588 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:42:05.595 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:42:05.595 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:42:05.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:42:05.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:42:05.662 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:42:05.672 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:42:05.672 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:42:05.676 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:42:05.679 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:42:05.716 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:05.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:05.903 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:05.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:06.064 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:42:06.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:42:06.144 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:06.190 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:06.247 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:06.331 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:06.373 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:42:06.373 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:42:06.438 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:06.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:42:06.534 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:06.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:42:06.659 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:42:06.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:42:06.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-04-26T21:42:06.732 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-04-26T21:42:06.734 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-04-26T21:42:06.772 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:42:06.806 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:42:06.836 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:42:07.270 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:42:07.278 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:42:07.278 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:42:07.297 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:42:07.307 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:42:07.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-04-26T21:42:07.400 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-04-26T21:42:07.404 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-04-26T21:42:07.463 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-04-26T21:42:07.465 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-04-26T21:42:07.529 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-04-26T21:42:07.531 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:42:07.532 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-04-26T21:42:07.582 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-04-26T21:42:07.582 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-04-26T21:42:07.612 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-04-26T21:42:07.613 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-04-26T21:42:07.640 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-04-26T21:42:07.640 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:42:07.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:42:07.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:07.708 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:42:07.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:07.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:42:07.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:42:07.765 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:42:07.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:42:07.914 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:42:07.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:42:07.983 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:42:07.993 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T21:42:07.999 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T21:42:08.146 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:42:08.162 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:42:08.163 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi077.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:47:18.074 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-04-26T21:47:18.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:47:18.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:47:18.189 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-26T21:47:18.347 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T21:47:18.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-26T21:47:18.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:47:18.419 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-04-26T21:47:18.501 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-26T21:47:18.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-26T21:47:18.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:47:18.574 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-26T21:47:18.662 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-26T21:47:18.663 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:47:18.674 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:47:18.674 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:47:18.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:47:18.682 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:47:18.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:47:18.685 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:47:18.685 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:47:18.714 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:47:18.728 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:47:18.730 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:47:18.744 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:47:18.752 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-04-26T21:47:18.763 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-04-26T21:47:18.768 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-04-26T21:47:18.775 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:47:18.776 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:47:18.778 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-04-26T21:47:18.790 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:47:18.792 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:47:18.804 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:47:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:47:18.842 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.845 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.856 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.862 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:18.898 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-04-26T21:47:18.932 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-04-26T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:47:19.011 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:47:19.157 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:47:19.166 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:47:19.166 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-04-26T21:47:19.167 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:19.167 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:19.167 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:19.167 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:47:19.168 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:47:19.176 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:47:19.176 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:47:19.176 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:47:19.177 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:47:19.182 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:47:19.182 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:47:19.182 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:47:19.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:47:19.184 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:47:19.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:47:19.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:47:19.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:47:19.332 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:47:19.332 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:47:19.405 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:47:19.405 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:47:19.409 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:47:19.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:47:19.506 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:47:19.506 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:47:19.506 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:47:19.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:47:19.532 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:47:19.573 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:47:19.573 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:47:19.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:47:19.583 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:47:19.583 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:47:19.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:47:19.598 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:47:19.608 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:47:19.608 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:47:19.667 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:47:19.668 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:47:19.673 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:47:19.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:47:19.738 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-04-26T21:47:19.740 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-04-26T21:47:19.749 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:47:19.801 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:47:19.813 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-04-26T21:47:19.815 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-04-26T21:47:19.918 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:47:19.966 INFO:teuthology.orchestra.run.smithi154.stdout:48 files removed 2024-04-26T21:47:19.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:47:20.003 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-04-26T21:47:20.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-04-26T21:47:20.038 INFO:teuthology.orchestra.run.smithi077.stdout:48 files removed 2024-04-26T21:47:20.066 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:47:20.349 INFO:teuthology.orchestra.run.smithi012.stdout:48 files removed 2024-04-26T21:47:20.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:47:20.718 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 180 kB/s | 84 kB 00:00 2024-04-26T21:47:20.725 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 209 kB/s | 84 kB 00:00 2024-04-26T21:47:21.077 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-26T21:47:21.143 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T21:47:21.224 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 219 kB/s | 84 kB 00:00 2024-04-26T21:47:21.435 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-26T21:47:21.577 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-26T21:47:21.699 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T21:47:22.169 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:47:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-26T21:47:22.477 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-04-26T21:47:22.931 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:47:23.987 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T21:47:24.229 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-26T21:47:25.066 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T21:47:29.296 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-26T21:47:29.404 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-26T21:47:29.549 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-26T21:47:30.822 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-04-26T21:47:30.863 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-26T21:47:31.266 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-26T21:47:31.915 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T21:47:32.910 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T21:47:33.460 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T21:47:36.956 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T21:47:37.973 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-26T21:47:39.133 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:39.133 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:39.144 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:47:39.145 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:47:39.146 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:47:39.147 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:47:39.148 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:47:39.150 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:47:39.151 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:47:39.214 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:47:39.241 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:47:39.242 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:47:39.243 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:47:39.243 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:47:39.243 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:47:39.247 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:47:39.248 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:47:39.249 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:47:39.250 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:47:39.251 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:47:39.252 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:47:39.253 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:47:39.254 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout:Install 148 Packages 2024-04-26T21:47:39.255 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 4 Packages 2024-04-26T21:47:39.256 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:47:39.256 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 222 M 2024-04-26T21:47:39.256 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-04-26T21:47:39.592 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T21:47:40.041 INFO:teuthology.orchestra.run.smithi154.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-26T21:47:40.099 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:40.099 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:40.110 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:47:40.111 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:47:40.112 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:47:40.113 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:47:40.114 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:47:40.116 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:47:40.117 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:47:40.178 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:47:40.202 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:47:40.203 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:47:40.204 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:47:40.205 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:47:40.206 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:47:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:47:40.208 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:47:40.209 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:47:40.210 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:47:40.211 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:47:40.212 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:47:40.213 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 4 Packages 2024-04-26T21:47:40.214 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:47:40.215 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 222 M 2024-04-26T21:47:40.215 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T21:47:40.241 INFO:teuthology.orchestra.run.smithi154.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.4 MB/s | 889 kB 00:00 2024-04-26T21:47:40.283 INFO:teuthology.orchestra.run.smithi154.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 3.4 MB/s | 145 kB 00:00 2024-04-26T21:47:40.508 INFO:teuthology.orchestra.run.smithi154.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 2.4 MB 00:00 2024-04-26T21:47:40.650 INFO:teuthology.orchestra.run.smithi154.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-26T21:47:40.767 INFO:teuthology.orchestra.run.smithi154.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 6.0 MB/s | 5.4 MB 00:00 2024-04-26T21:47:40.984 INFO:teuthology.orchestra.run.smithi154.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-26T21:47:41.266 INFO:teuthology.orchestra.run.smithi077.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-26T21:47:41.968 INFO:teuthology.orchestra.run.smithi154.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 11 MB/s | 11 MB 00:00 2024-04-26T21:47:42.019 INFO:teuthology.orchestra.run.smithi154.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 505 kB/s | 25 kB 00:00 2024-04-26T21:47:42.033 INFO:teuthology.orchestra.run.smithi077.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 889 kB 00:00 2024-04-26T21:47:42.125 INFO:teuthology.orchestra.run.smithi077.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.5 MB/s | 145 kB 00:00 2024-04-26T21:47:42.144 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:42.146 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:47:42.170 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:47:42.171 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:47:42.173 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:47:42.174 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:47:42.177 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:47:42.181 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:47:42.182 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:47:42.259 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-26T21:47:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:47:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:47:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:47:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:47:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:47:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:47:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:47:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:47:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:47:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:47:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:47:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:47:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:47:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:47:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout:Install 148 Packages 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 4 Packages 2024-04-26T21:47:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 222 M 2024-04-26T21:47:42.303 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-04-26T21:47:42.660 INFO:teuthology.orchestra.run.smithi154.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 8.9 MB/s | 17 MB 00:01 2024-04-26T21:47:42.752 INFO:teuthology.orchestra.run.smithi154.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 225 kB 00:00 2024-04-26T21:47:42.861 INFO:teuthology.orchestra.run.smithi154.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 249 kB/s | 27 kB 00:00 2024-04-26T21:47:42.953 INFO:teuthology.orchestra.run.smithi154.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 346 kB/s | 32 kB 00:00 2024-04-26T21:47:43.018 INFO:teuthology.orchestra.run.smithi077.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-04-26T21:47:43.128 INFO:teuthology.orchestra.run.smithi154.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.2 MB/s | 747 kB 00:00 2024-04-26T21:47:43.237 INFO:teuthology.orchestra.run.smithi154.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 163 kB 00:00 2024-04-26T21:47:43.337 INFO:teuthology.orchestra.run.smithi154.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.2 MB/s | 127 kB 00:00 2024-04-26T21:47:43.393 INFO:teuthology.orchestra.run.smithi012.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-26T21:47:43.470 INFO:teuthology.orchestra.run.smithi154.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 3.7 MB/s | 501 kB 00:00 2024-04-26T21:47:43.651 INFO:teuthology.orchestra.run.smithi077.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-04-26T21:47:44.529 INFO:teuthology.orchestra.run.smithi154.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 5.1 MB/s | 5.4 MB 00:01 2024-04-26T21:47:44.679 INFO:teuthology.orchestra.run.smithi154.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 301 kB/s | 45 kB 00:00 2024-04-26T21:47:44.701 INFO:teuthology.orchestra.run.smithi077.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-26T21:47:44.788 INFO:teuthology.orchestra.run.smithi154.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:47:44.888 INFO:teuthology.orchestra.run.smithi154.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.6 MB/s | 161 kB 00:00 2024-04-26T21:47:44.997 INFO:teuthology.orchestra.run.smithi154.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.9 MB/s | 321 kB 00:00 2024-04-26T21:47:45.068 INFO:teuthology.orchestra.run.smithi012.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 531 kB/s | 889 kB 00:01 2024-04-26T21:47:45.122 INFO:teuthology.orchestra.run.smithi154.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T21:47:45.231 INFO:teuthology.orchestra.run.smithi154.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 918 kB/s | 99 kB 00:00 2024-04-26T21:47:45.294 INFO:teuthology.orchestra.run.smithi012.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 648 kB/s | 145 kB 00:00 2024-04-26T21:47:45.323 INFO:teuthology.orchestra.run.smithi154.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 932 kB/s | 85 kB 00:00 2024-04-26T21:47:45.810 INFO:teuthology.orchestra.run.smithi077.stdout:(7/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 4.3 MB/s | 21 MB 00:04 2024-04-26T21:47:45.877 INFO:teuthology.orchestra.run.smithi077.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-04-26T21:47:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 336 kB/s | 25 kB 00:00 2024-04-26T21:47:46.015 INFO:teuthology.orchestra.run.smithi154.stdout:(26/152): ceph-common-19.0.0-3306.gbc612ffb.el9 3.4 MB/s | 21 MB 00:06 2024-04-26T21:47:46.073 INFO:teuthology.orchestra.run.smithi154.stdout:(27/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.1 MB/s | 3.1 MB 00:00 2024-04-26T21:47:46.115 INFO:teuthology.orchestra.run.smithi154.stdout:(28/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-26T21:47:46.165 INFO:teuthology.orchestra.run.smithi154.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 372 kB/s | 34 kB 00:00 2024-04-26T21:47:46.215 INFO:teuthology.orchestra.run.smithi154.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T21:47:46.461 INFO:teuthology.orchestra.run.smithi012.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-26T21:47:46.874 INFO:teuthology.orchestra.run.smithi154.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.0 MB/s | 3.5 MB 00:00 2024-04-26T21:47:46.974 INFO:teuthology.orchestra.run.smithi154.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.4 MB/s | 249 kB 00:00 2024-04-26T21:47:47.077 INFO:teuthology.orchestra.run.smithi154.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 484 kB/s | 49 kB 00:00 2024-04-26T21:47:47.169 INFO:teuthology.orchestra.run.smithi154.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 179 kB/s | 16 kB 00:00 2024-04-26T21:47:47.252 INFO:teuthology.orchestra.run.smithi154.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 273 kB 00:00 2024-04-26T21:47:47.269 INFO:teuthology.orchestra.run.smithi077.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 7.3 MB/s | 11 MB 00:01 2024-04-26T21:47:47.269 INFO:teuthology.orchestra.run.smithi012.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-26T21:47:47.352 INFO:teuthology.orchestra.run.smithi077.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.6 MB/s | 225 kB 00:00 2024-04-26T21:47:47.403 INFO:teuthology.orchestra.run.smithi154.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T21:47:47.419 INFO:teuthology.orchestra.run.smithi077.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 406 kB/s | 27 kB 00:00 2024-04-26T21:47:47.478 INFO:teuthology.orchestra.run.smithi077.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 543 kB/s | 32 kB 00:00 2024-04-26T21:47:47.478 INFO:teuthology.orchestra.run.smithi154.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 244 kB/s | 18 kB 00:00 2024-04-26T21:47:47.603 INFO:teuthology.orchestra.run.smithi077.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 747 kB 00:00 2024-04-26T21:47:47.628 INFO:teuthology.orchestra.run.smithi154.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-04-26T21:47:47.662 INFO:teuthology.orchestra.run.smithi077.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 163 kB 00:00 2024-04-26T21:47:47.712 INFO:teuthology.orchestra.run.smithi077.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:47:47.796 INFO:teuthology.orchestra.run.smithi077.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.9 MB/s | 501 kB 00:00 2024-04-26T21:47:47.802 INFO:teuthology.orchestra.run.smithi012.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 964 kB/s | 2.4 MB 00:02 2024-04-26T21:47:48.079 INFO:teuthology.orchestra.run.smithi154.stdout:(39/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 8.1 MB/s | 49 MB 00:06 2024-04-26T21:47:48.179 INFO:teuthology.orchestra.run.smithi154.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-26T21:47:48.279 INFO:teuthology.orchestra.run.smithi154.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-26T21:47:48.312 INFO:teuthology.orchestra.run.smithi077.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 5.4 MB 00:00 2024-04-26T21:47:48.321 INFO:teuthology.orchestra.run.smithi154.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 809 kB/s | 195 kB 00:00 2024-04-26T21:47:48.355 INFO:teuthology.orchestra.run.smithi077.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 1.1 MB/s | 45 kB 00:00 2024-04-26T21:47:48.388 INFO:teuthology.orchestra.run.smithi154.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T21:47:48.405 INFO:teuthology.orchestra.run.smithi077.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.6 MB/s | 135 kB 00:00 2024-04-26T21:47:48.455 INFO:teuthology.orchestra.run.smithi077.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 3.2 MB/s | 161 kB 00:00 2024-04-26T21:47:48.471 INFO:teuthology.orchestra.run.smithi154.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T21:47:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 6.3 MB/s | 321 kB 00:00 2024-04-26T21:47:48.538 INFO:teuthology.orchestra.run.smithi154.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T21:47:48.556 INFO:teuthology.orchestra.run.smithi077.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 5.8 MB/s | 298 kB 00:00 2024-04-26T21:47:48.569 INFO:teuthology.orchestra.run.smithi012.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-04-26T21:47:48.606 INFO:teuthology.orchestra.run.smithi077.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 99 kB 00:00 2024-04-26T21:47:48.622 INFO:teuthology.orchestra.run.smithi154.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T21:47:48.656 INFO:teuthology.orchestra.run.smithi077.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-26T21:47:48.664 INFO:teuthology.orchestra.run.smithi154.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-04-26T21:47:48.831 INFO:teuthology.orchestra.run.smithi154.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-26T21:47:48.890 INFO:teuthology.orchestra.run.smithi154.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T21:47:48.973 INFO:teuthology.orchestra.run.smithi154.stdout:(50/152): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-04-26T21:47:48.998 INFO:teuthology.orchestra.run.smithi154.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-26T21:47:49.015 INFO:teuthology.orchestra.run.smithi077.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.6 MB/s | 3.1 MB 00:00 2024-04-26T21:47:49.049 INFO:teuthology.orchestra.run.smithi154.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-26T21:47:49.057 INFO:teuthology.orchestra.run.smithi077.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 171 kB 00:00 2024-04-26T21:47:49.074 INFO:teuthology.orchestra.run.smithi154.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-04-26T21:47:49.091 INFO:teuthology.orchestra.run.smithi077.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 1.0 MB/s | 34 kB 00:00 2024-04-26T21:47:49.132 INFO:teuthology.orchestra.run.smithi154.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-26T21:47:49.141 INFO:teuthology.orchestra.run.smithi077.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T21:47:49.183 INFO:teuthology.orchestra.run.smithi154.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-26T21:47:49.241 INFO:teuthology.orchestra.run.smithi154.stdout:(56/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-26T21:47:49.277 INFO:teuthology.orchestra.run.smithi154.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-26T21:47:49.309 INFO:teuthology.orchestra.run.smithi154.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-26T21:47:49.459 INFO:teuthology.orchestra.run.smithi154.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-26T21:47:49.500 INFO:teuthology.orchestra.run.smithi077.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 9.9 MB/s | 3.5 MB 00:00 2024-04-26T21:47:49.517 INFO:teuthology.orchestra.run.smithi154.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-04-26T21:47:49.534 INFO:teuthology.orchestra.run.smithi154.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-26T21:47:49.568 INFO:teuthology.orchestra.run.smithi154.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-04-26T21:47:49.635 INFO:teuthology.orchestra.run.smithi154.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-26T21:47:49.660 INFO:teuthology.orchestra.run.smithi154.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T21:47:49.686 INFO:teuthology.orchestra.run.smithi154.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-26T21:47:49.903 INFO:teuthology.orchestra.run.smithi154.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-04-26T21:47:50.084 INFO:teuthology.orchestra.run.smithi077.stdout:(31/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-26T21:47:50.167 INFO:teuthology.orchestra.run.smithi077.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.9 MB/s | 249 kB 00:00 2024-04-26T21:47:50.276 INFO:teuthology.orchestra.run.smithi077.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-04-26T21:47:50.301 INFO:teuthology.orchestra.run.smithi077.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 369 kB/s | 49 kB 00:00 2024-04-26T21:47:50.320 INFO:teuthology.orchestra.run.smithi154.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-04-26T21:47:50.326 INFO:teuthology.orchestra.run.smithi077.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 328 kB/s | 16 kB 00:00 2024-04-26T21:47:50.376 INFO:teuthology.orchestra.run.smithi077.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 273 kB 00:00 2024-04-26T21:47:50.419 INFO:teuthology.orchestra.run.smithi077.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 437 kB/s | 18 kB 00:00 2024-04-26T21:47:50.461 INFO:teuthology.orchestra.run.smithi077.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-26T21:47:50.478 INFO:teuthology.orchestra.run.smithi154.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-26T21:47:50.537 INFO:teuthology.orchestra.run.smithi154.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-26T21:47:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 5.3 MB/s | 11 MB 00:01 2024-04-26T21:47:50.571 INFO:teuthology.orchestra.run.smithi154.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-04-26T21:47:50.594 INFO:teuthology.orchestra.run.smithi077.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-04-26T21:47:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 515 kB/s | 25 kB 00:00 2024-04-26T21:47:50.728 INFO:teuthology.orchestra.run.smithi077.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-04-26T21:47:50.796 INFO:teuthology.orchestra.run.smithi154.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-26T21:47:50.828 INFO:teuthology.orchestra.run.smithi077.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-04-26T21:47:50.863 INFO:teuthology.orchestra.run.smithi154.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T21:47:50.878 INFO:teuthology.orchestra.run.smithi077.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-26T21:47:50.930 INFO:teuthology.orchestra.run.smithi154.stdout:(73/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.7 MB/s | 4.7 MB 00:01 2024-04-26T21:47:50.970 INFO:teuthology.orchestra.run.smithi077.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-26T21:47:51.022 INFO:teuthology.orchestra.run.smithi154.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T21:47:51.029 INFO:teuthology.orchestra.run.smithi077.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-26T21:47:51.054 INFO:teuthology.orchestra.run.smithi077.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T21:47:51.096 INFO:teuthology.orchestra.run.smithi077.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T21:47:51.138 INFO:teuthology.orchestra.run.smithi154.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-26T21:47:51.188 INFO:teuthology.orchestra.run.smithi077.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-04-26T21:47:51.239 INFO:teuthology.orchestra.run.smithi154.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 8.2 MB/s | 5.8 MB 00:00 2024-04-26T21:47:51.264 INFO:teuthology.orchestra.run.smithi077.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-26T21:47:51.264 INFO:teuthology.orchestra.run.smithi154.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-04-26T21:47:51.298 INFO:teuthology.orchestra.run.smithi154.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-26T21:47:51.314 INFO:teuthology.orchestra.run.smithi077.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T21:47:51.323 INFO:teuthology.orchestra.run.smithi154.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:47:51.357 INFO:teuthology.orchestra.run.smithi154.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-26T21:47:51.364 INFO:teuthology.orchestra.run.smithi077.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-26T21:47:51.415 INFO:teuthology.orchestra.run.smithi154.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-26T21:47:51.448 INFO:teuthology.orchestra.run.smithi077.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-26T21:47:51.482 INFO:teuthology.orchestra.run.smithi154.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T21:47:51.506 INFO:teuthology.orchestra.run.smithi077.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-26T21:47:51.532 INFO:teuthology.orchestra.run.smithi077.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-04-26T21:47:51.533 INFO:teuthology.orchestra.run.smithi154.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:47:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-26T21:47:51.591 INFO:teuthology.orchestra.run.smithi154.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 728 kB/s | 42 kB 00:00 2024-04-26T21:47:51.658 INFO:teuthology.orchestra.run.smithi077.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-26T21:47:51.666 INFO:teuthology.orchestra.run.smithi154.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T21:47:51.725 INFO:teuthology.orchestra.run.smithi077.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-26T21:47:51.758 INFO:teuthology.orchestra.run.smithi154.stdout:(86/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.7 MB/s | 6.0 MB 00:00 2024-04-26T21:47:51.775 INFO:teuthology.orchestra.run.smithi077.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-04-26T21:47:51.784 INFO:teuthology.orchestra.run.smithi154.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-26T21:47:51.826 INFO:teuthology.orchestra.run.smithi154.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T21:47:51.851 INFO:teuthology.orchestra.run.smithi154.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 763 kB/s | 51 kB 00:00 2024-04-26T21:47:51.884 INFO:teuthology.orchestra.run.smithi154.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 918 kB/s | 54 kB 00:00 2024-04-26T21:47:51.908 INFO:teuthology.orchestra.run.smithi077.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-04-26T21:47:51.952 INFO:teuthology.orchestra.run.smithi154.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-04-26T21:47:52.009 INFO:teuthology.orchestra.run.smithi077.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-26T21:47:52.010 INFO:teuthology.orchestra.run.smithi154.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-04-26T21:47:52.077 INFO:teuthology.orchestra.run.smithi154.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T21:47:52.084 INFO:teuthology.orchestra.run.smithi077.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-04-26T21:47:52.109 INFO:teuthology.orchestra.run.smithi077.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-04-26T21:47:52.135 INFO:teuthology.orchestra.run.smithi077.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-26T21:47:52.136 INFO:teuthology.orchestra.run.smithi154.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-26T21:47:52.168 INFO:teuthology.orchestra.run.smithi077.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-26T21:47:52.194 INFO:teuthology.orchestra.run.smithi077.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-26T21:47:52.211 INFO:teuthology.orchestra.run.smithi154.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-04-26T21:47:52.227 INFO:teuthology.orchestra.run.smithi077.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-26T21:47:52.286 INFO:teuthology.orchestra.run.smithi154.stdout:(96/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.3 MB/s | 5.1 MB 00:00 2024-04-26T21:47:52.312 INFO:teuthology.orchestra.run.smithi154.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-26T21:47:52.337 INFO:teuthology.orchestra.run.smithi154.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:47:52.371 INFO:teuthology.orchestra.run.smithi154.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T21:47:52.412 INFO:teuthology.orchestra.run.smithi154.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-04-26T21:47:52.429 INFO:teuthology.orchestra.run.smithi154.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-26T21:47:52.613 INFO:teuthology.orchestra.run.smithi154.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-04-26T21:47:52.719 INFO:teuthology.orchestra.run.smithi077.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.0 MB/s | 4.7 MB 00:00 2024-04-26T21:47:52.786 INFO:teuthology.orchestra.run.smithi077.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-04-26T21:47:52.911 INFO:teuthology.orchestra.run.smithi077.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T21:47:52.954 INFO:teuthology.orchestra.run.smithi077.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-26T21:47:52.979 INFO:teuthology.orchestra.run.smithi077.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T21:47:53.071 INFO:teuthology.orchestra.run.smithi077.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T21:47:53.072 INFO:teuthology.orchestra.run.smithi154.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-26T21:47:53.129 INFO:teuthology.orchestra.run.smithi077.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T21:47:53.264 INFO:teuthology.orchestra.run.smithi154.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-04-26T21:47:53.297 INFO:teuthology.orchestra.run.smithi154.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 802 kB/s | 548 kB 00:00 2024-04-26T21:47:53.346 INFO:teuthology.orchestra.run.smithi077.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-26T21:47:53.405 INFO:teuthology.orchestra.run.smithi077.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T21:47:53.464 INFO:teuthology.orchestra.run.smithi077.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-26T21:47:53.489 INFO:teuthology.orchestra.run.smithi154.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 741 kB/s | 309 kB 00:00 2024-04-26T21:47:53.523 INFO:teuthology.orchestra.run.smithi154.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-26T21:47:53.547 INFO:teuthology.orchestra.run.smithi077.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-26T21:47:53.572 INFO:teuthology.orchestra.run.smithi077.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-04-26T21:47:53.573 INFO:teuthology.orchestra.run.smithi154.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-04-26T21:47:53.615 INFO:teuthology.orchestra.run.smithi077.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 284 kB/s | 19 kB 00:00 2024-04-26T21:47:53.640 INFO:teuthology.orchestra.run.smithi077.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-26T21:47:53.674 INFO:teuthology.orchestra.run.smithi077.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-04-26T21:47:53.724 INFO:teuthology.orchestra.run.smithi077.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-26T21:47:53.791 INFO:teuthology.orchestra.run.smithi077.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T21:47:53.823 INFO:teuthology.orchestra.run.smithi154.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2024-04-26T21:47:53.850 INFO:teuthology.orchestra.run.smithi077.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:47:53.908 INFO:teuthology.orchestra.run.smithi077.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-26T21:47:53.924 INFO:teuthology.orchestra.run.smithi154.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 121 kB/s | 49 kB 00:00 2024-04-26T21:47:53.949 INFO:teuthology.orchestra.run.smithi154.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-26T21:47:53.975 INFO:teuthology.orchestra.run.smithi077.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-26T21:47:54.001 INFO:teuthology.orchestra.run.smithi077.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-26T21:47:54.026 INFO:teuthology.orchestra.run.smithi077.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T21:47:54.068 INFO:teuthology.orchestra.run.smithi077.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T21:47:54.093 INFO:teuthology.orchestra.run.smithi077.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-26T21:47:54.118 INFO:teuthology.orchestra.run.smithi077.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T21:47:54.186 INFO:teuthology.orchestra.run.smithi077.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-04-26T21:47:54.252 INFO:teuthology.orchestra.run.smithi077.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 664 kB/s | 44 kB 00:00 2024-04-26T21:47:54.319 INFO:teuthology.orchestra.run.smithi077.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T21:47:54.378 INFO:teuthology.orchestra.run.smithi077.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-26T21:47:54.428 INFO:teuthology.orchestra.run.smithi077.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-26T21:47:54.457 INFO:teuthology.orchestra.run.smithi154.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-04-26T21:47:54.487 INFO:teuthology.orchestra.run.smithi077.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-26T21:47:54.546 INFO:teuthology.orchestra.run.smithi077.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-04-26T21:47:54.577 INFO:teuthology.orchestra.run.smithi012.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.8 MB/s | 21 MB 00:11 2024-04-26T21:47:54.596 INFO:teuthology.orchestra.run.smithi077.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T21:47:54.624 INFO:teuthology.orchestra.run.smithi154.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-04-26T21:47:54.646 INFO:teuthology.orchestra.run.smithi077.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:47:54.705 INFO:teuthology.orchestra.run.smithi077.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-04-26T21:47:54.725 INFO:teuthology.orchestra.run.smithi154.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 163 kB/s | 43 kB 00:00 2024-04-26T21:47:54.750 INFO:teuthology.orchestra.run.smithi154.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-26T21:47:54.811 INFO:teuthology.orchestra.run.smithi012.stdout:(11/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 2.4 MB/s | 17 MB 00:07 2024-04-26T21:47:54.817 INFO:teuthology.orchestra.run.smithi154.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-26T21:47:54.844 INFO:teuthology.orchestra.run.smithi012.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 846 kB/s | 225 kB 00:00 2024-04-26T21:47:54.872 INFO:teuthology.orchestra.run.smithi077.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-26T21:47:54.878 INFO:teuthology.orchestra.run.smithi012.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 412 kB/s | 27 kB 00:00 2024-04-26T21:47:54.892 INFO:teuthology.orchestra.run.smithi154.stdout:(117/152): libarrow-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-04-26T21:47:54.911 INFO:teuthology.orchestra.run.smithi012.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 479 kB/s | 32 kB 00:00 2024-04-26T21:47:54.917 INFO:teuthology.orchestra.run.smithi154.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-26T21:47:54.987 INFO:teuthology.orchestra.run.smithi012.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 163 kB 00:00 2024-04-26T21:47:54.998 INFO:teuthology.orchestra.run.smithi154.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-26T21:47:55.027 INFO:teuthology.orchestra.run.smithi154.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-26T21:47:55.052 INFO:teuthology.orchestra.run.smithi154.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-26T21:47:55.054 INFO:teuthology.orchestra.run.smithi012.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T21:47:55.104 INFO:teuthology.orchestra.run.smithi012.stdout:(17/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.2 MB/s | 747 kB 00:00 2024-04-26T21:47:55.110 INFO:teuthology.orchestra.run.smithi154.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-26T21:47:55.152 INFO:teuthology.orchestra.run.smithi154.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-26T21:47:55.219 INFO:teuthology.orchestra.run.smithi154.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-26T21:47:55.245 INFO:teuthology.orchestra.run.smithi154.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-26T21:47:55.303 INFO:teuthology.orchestra.run.smithi154.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-26T21:47:55.328 INFO:teuthology.orchestra.run.smithi154.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-26T21:47:55.364 INFO:teuthology.orchestra.run.smithi077.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-04-26T21:47:55.379 INFO:teuthology.orchestra.run.smithi154.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-26T21:47:55.479 INFO:teuthology.orchestra.run.smithi154.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-26T21:47:55.596 INFO:teuthology.orchestra.run.smithi154.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-04-26T21:47:55.630 INFO:teuthology.orchestra.run.smithi154.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-04-26T21:47:55.646 INFO:teuthology.orchestra.run.smithi012.stdout:(18/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 9.7 MB/s | 49 MB 00:05 2024-04-26T21:47:55.681 INFO:teuthology.orchestra.run.smithi077.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-26T21:47:55.688 INFO:teuthology.orchestra.run.smithi012.stdout:(19/152): libradosstriper1-19.0.0-3306.gbc612ff 792 kB/s | 501 kB 00:00 2024-04-26T21:47:55.688 INFO:teuthology.orchestra.run.smithi154.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-26T21:47:55.705 INFO:teuthology.orchestra.run.smithi012.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 788 kB/s | 45 kB 00:00 2024-04-26T21:47:55.731 INFO:teuthology.orchestra.run.smithi154.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T21:47:55.734 INFO:teuthology.orchestra.run.smithi077.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.8 MB/s | 309 kB 00:00 2024-04-26T21:47:55.772 INFO:teuthology.orchestra.run.smithi012.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T21:47:55.781 INFO:teuthology.orchestra.run.smithi154.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-26T21:47:55.792 INFO:teuthology.orchestra.run.smithi077.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-04-26T21:47:55.830 INFO:teuthology.orchestra.run.smithi012.stdout:(22/152): python3-ceph-common-19.0.0-3306.gbc61 957 kB/s | 135 kB 00:00 2024-04-26T21:47:55.831 INFO:teuthology.orchestra.run.smithi154.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 899 kB/s | 90 kB 00:00 2024-04-26T21:47:55.864 INFO:teuthology.orchestra.run.smithi012.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.5 MB/s | 321 kB 00:00 2024-04-26T21:47:55.873 INFO:teuthology.orchestra.run.smithi154.stdout:(136/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-26T21:47:55.890 INFO:teuthology.orchestra.run.smithi154.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-26T21:47:55.914 INFO:teuthology.orchestra.run.smithi012.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 99 kB 00:00 2024-04-26T21:47:55.940 INFO:teuthology.orchestra.run.smithi154.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-26T21:47:55.965 INFO:teuthology.orchestra.run.smithi012.stdout:(25/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.2 MB/s | 298 kB 00:00 2024-04-26T21:47:55.982 INFO:teuthology.orchestra.run.smithi154.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T21:47:55.990 INFO:teuthology.orchestra.run.smithi012.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T21:47:56.007 INFO:teuthology.orchestra.run.smithi154.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-26T21:47:56.025 INFO:teuthology.orchestra.run.smithi154.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-04-26T21:47:56.026 INFO:teuthology.orchestra.run.smithi077.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-26T21:47:56.048 INFO:teuthology.orchestra.run.smithi012.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T21:47:56.058 INFO:teuthology.orchestra.run.smithi154.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-26T21:47:56.076 INFO:teuthology.orchestra.run.smithi077.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-04-26T21:47:56.107 INFO:teuthology.orchestra.run.smithi012.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 597 kB/s | 34 kB 00:00 2024-04-26T21:47:56.126 INFO:teuthology.orchestra.run.smithi077.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-26T21:47:56.166 INFO:teuthology.orchestra.run.smithi012.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.4 MB/s | 142 kB 00:00 2024-04-26T21:47:56.166 INFO:teuthology.orchestra.run.smithi154.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 631 kB/s | 90 kB 00:00 2024-04-26T21:47:56.176 INFO:teuthology.orchestra.run.smithi077.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-26T21:47:56.192 INFO:teuthology.orchestra.run.smithi154.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-26T21:47:56.235 INFO:teuthology.orchestra.run.smithi077.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-26T21:47:56.259 INFO:teuthology.orchestra.run.smithi154.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-26T21:47:56.284 INFO:teuthology.orchestra.run.smithi154.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2024-04-26T21:47:56.310 INFO:teuthology.orchestra.run.smithi077.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-26T21:47:56.318 INFO:teuthology.orchestra.run.smithi154.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-26T21:47:56.361 INFO:teuthology.orchestra.run.smithi077.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-26T21:47:56.416 INFO:teuthology.orchestra.run.smithi012.stdout:(30/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-04-26T21:47:56.419 INFO:teuthology.orchestra.run.smithi077.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-04-26T21:47:56.470 INFO:teuthology.orchestra.run.smithi077.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 646 kB/s | 32 kB 00:00 2024-04-26T21:47:56.520 INFO:teuthology.orchestra.run.smithi077.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-26T21:47:56.553 INFO:teuthology.orchestra.run.smithi077.stdout:(116/152): protobuf-compiler-3.14.0-13.el9.x86_ 514 kB/s | 863 kB 00:01 2024-04-26T21:47:56.579 INFO:teuthology.orchestra.run.smithi077.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-26T21:47:56.716 INFO:teuthology.orchestra.run.smithi012.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 6.4 MB/s | 3.5 MB 00:00 2024-04-26T21:47:56.735 INFO:teuthology.orchestra.run.smithi154.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.5 MB/s | 3.4 MB 00:00 2024-04-26T21:47:56.802 INFO:teuthology.orchestra.run.smithi154.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-26T21:47:56.808 INFO:teuthology.orchestra.run.smithi012.stdout:(32/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.7 MB/s | 3.1 MB 00:00 2024-04-26T21:47:56.842 INFO:teuthology.orchestra.run.smithi012.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.0 MB/s | 249 kB 00:00 2024-04-26T21:47:56.875 INFO:teuthology.orchestra.run.smithi012.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 749 kB/s | 49 kB 00:00 2024-04-26T21:47:56.901 INFO:teuthology.orchestra.run.smithi012.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 285 kB/s | 16 kB 00:00 2024-04-26T21:47:56.968 INFO:teuthology.orchestra.run.smithi012.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T21:47:56.977 INFO:teuthology.orchestra.run.smithi154.stdout:(150/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-26T21:47:56.988 INFO:teuthology.orchestra.run.smithi077.stdout:(118/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 4.5 MB/s | 49 MB 00:11 2024-04-26T21:47:57.011 INFO:teuthology.orchestra.run.smithi154.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-04-26T21:47:57.013 INFO:teuthology.orchestra.run.smithi077.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 782 kB/s | 358 kB 00:00 2024-04-26T21:47:57.026 INFO:teuthology.orchestra.run.smithi012.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-26T21:47:57.038 INFO:teuthology.orchestra.run.smithi077.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 660 kB/s | 303 kB 00:00 2024-04-26T21:47:57.051 INFO:teuthology.orchestra.run.smithi012.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 221 kB/s | 18 kB 00:00 2024-04-26T21:47:57.063 INFO:teuthology.orchestra.run.smithi077.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 478 kB/s | 35 kB 00:00 2024-04-26T21:47:57.097 INFO:teuthology.orchestra.run.smithi077.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-26T21:47:57.118 INFO:teuthology.orchestra.run.smithi012.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 886 kB/s | 80 kB 00:00 2024-04-26T21:47:57.177 INFO:teuthology.orchestra.run.smithi012.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-26T21:47:57.202 INFO:teuthology.orchestra.run.smithi154.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 2.4 MB 00:00 2024-04-26T21:47:57.205 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:47:57.206 INFO:teuthology.orchestra.run.smithi077.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T21:47:57.206 INFO:teuthology.orchestra.run.smithi154.stdout:Total 12 MB/s | 222 MB 00:17 2024-04-26T21:47:57.219 INFO:teuthology.orchestra.run.smithi012.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T21:47:57.239 INFO:teuthology.orchestra.run.smithi077.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 824 kB/s | 144 kB 00:00 2024-04-26T21:47:57.256 INFO:teuthology.orchestra.run.smithi077.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 68 kB/s | 11 kB 00:00 2024-04-26T21:47:57.269 INFO:teuthology.orchestra.run.smithi012.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T21:47:57.281 INFO:teuthology.orchestra.run.smithi077.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-26T21:47:57.306 INFO:teuthology.orchestra.run.smithi077.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-26T21:47:57.332 INFO:teuthology.orchestra.run.smithi077.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-26T21:47:57.337 INFO:teuthology.orchestra.run.smithi012.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T21:47:57.348 INFO:teuthology.orchestra.run.smithi077.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-26T21:47:57.391 INFO:teuthology.orchestra.run.smithi077.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-26T21:47:57.407 INFO:teuthology.orchestra.run.smithi077.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-26T21:47:57.433 INFO:teuthology.orchestra.run.smithi077.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-26T21:47:57.450 INFO:teuthology.orchestra.run.smithi077.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-26T21:47:57.462 INFO:teuthology.orchestra.run.smithi012.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-26T21:47:57.492 INFO:teuthology.orchestra.run.smithi077.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-26T21:47:57.509 INFO:teuthology.orchestra.run.smithi077.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-26T21:47:57.528 INFO:teuthology.orchestra.run.smithi012.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-26T21:47:57.534 INFO:teuthology.orchestra.run.smithi077.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-26T21:47:57.559 INFO:teuthology.orchestra.run.smithi077.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-26T21:47:57.585 INFO:teuthology.orchestra.run.smithi077.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-26T21:47:57.610 INFO:teuthology.orchestra.run.smithi077.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T21:47:57.635 INFO:teuthology.orchestra.run.smithi077.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 788 kB/s | 59 kB 00:00 2024-04-26T21:47:57.660 INFO:teuthology.orchestra.run.smithi077.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-04-26T21:47:57.671 INFO:teuthology.orchestra.run.smithi012.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2024-04-26T21:47:57.685 INFO:teuthology.orchestra.run.smithi077.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-26T21:47:57.711 INFO:teuthology.orchestra.run.smithi077.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:47:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-04-26T21:47:57.745 INFO:teuthology.orchestra.run.smithi077.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-26T21:47:57.763 INFO:teuthology.orchestra.run.smithi012.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 921 kB/s | 215 kB 00:00 2024-04-26T21:47:57.778 INFO:teuthology.orchestra.run.smithi077.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-26T21:47:57.803 INFO:teuthology.orchestra.run.smithi077.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-26T21:47:57.829 INFO:teuthology.orchestra.run.smithi077.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-04-26T21:47:57.896 INFO:teuthology.orchestra.run.smithi077.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T21:47:57.988 INFO:teuthology.orchestra.run.smithi012.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 477 kB/s | 107 kB 00:00 2024-04-26T21:47:58.038 INFO:teuthology.orchestra.run.smithi012.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 438 kB/s | 21 kB 00:00 2024-04-26T21:47:58.146 INFO:teuthology.orchestra.run.smithi077.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-04-26T21:47:58.180 INFO:teuthology.orchestra.run.smithi012.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-26T21:47:58.239 INFO:teuthology.orchestra.run.smithi012.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 569 kB/s | 33 kB 00:00 2024-04-26T21:47:58.272 INFO:teuthology.orchestra.run.smithi012.stdout:(53/152): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-26T21:47:58.296 INFO:teuthology.orchestra.run.smithi077.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-26T21:47:58.318 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-04-26T21:47:58.347 INFO:teuthology.orchestra.run.smithi012.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 246 kB/s | 18 kB 00:00 2024-04-26T21:47:58.380 INFO:teuthology.orchestra.run.smithi077.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-26T21:47:58.414 INFO:teuthology.orchestra.run.smithi012.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 169 kB/s | 11 kB 00:00 2024-04-26T21:47:58.462 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-04-26T21:47:58.462 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-04-26T21:47:58.465 INFO:teuthology.orchestra.run.smithi012.stdout:(56/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-26T21:47:58.497 INFO:teuthology.orchestra.run.smithi077.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.9 MB/s | 3.4 MB 00:00 2024-04-26T21:47:58.501 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:47:58.501 INFO:teuthology.orchestra.run.smithi077.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-26T21:47:58.557 INFO:teuthology.orchestra.run.smithi012.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 498 kB/s | 45 kB 00:00 2024-04-26T21:47:58.582 INFO:teuthology.orchestra.run.smithi012.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 964 kB/s | 160 kB 00:00 2024-04-26T21:47:58.849 INFO:teuthology.orchestra.run.smithi012.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-04-26T21:47:58.891 INFO:teuthology.orchestra.run.smithi012.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-26T21:47:58.925 INFO:teuthology.orchestra.run.smithi012.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-26T21:47:59.008 INFO:teuthology.orchestra.run.smithi012.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-04-26T21:47:59.033 INFO:teuthology.orchestra.run.smithi012.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 335 kB/s | 47 kB 00:00 2024-04-26T21:47:59.075 INFO:teuthology.orchestra.run.smithi012.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-26T21:47:59.101 INFO:teuthology.orchestra.run.smithi012.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 532 kB/s | 35 kB 00:00 2024-04-26T21:47:59.126 INFO:teuthology.orchestra.run.smithi012.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-26T21:47:59.368 INFO:teuthology.orchestra.run.smithi012.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-04-26T21:47:59.526 INFO:teuthology.orchestra.run.smithi012.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-04-26T21:47:59.594 INFO:teuthology.orchestra.run.smithi012.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T21:47:59.630 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T21:47:59.776 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T21:47:59.776 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T21:47:59.977 INFO:teuthology.orchestra.run.smithi012.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.3 MB/s | 4.7 MB 00:00 2024-04-26T21:48:00.102 INFO:teuthology.orchestra.run.smithi012.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-04-26T21:48:00.219 INFO:teuthology.orchestra.run.smithi012.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 9.3 MB/s | 5.8 MB 00:00 2024-04-26T21:48:00.231 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-04-26T21:48:00.231 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-04-26T21:48:00.244 INFO:teuthology.orchestra.run.smithi012.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 621 kB/s | 87 kB 00:00 2024-04-26T21:48:00.295 INFO:teuthology.orchestra.run.smithi012.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-04-26T21:48:00.328 INFO:teuthology.orchestra.run.smithi012.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T21:48:00.370 INFO:teuthology.orchestra.run.smithi012.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-04-26T21:48:00.395 INFO:teuthology.orchestra.run.smithi012.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 392 kB/s | 26 kB 00:00 2024-04-26T21:48:00.429 INFO:teuthology.orchestra.run.smithi012.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-26T21:48:00.488 INFO:teuthology.orchestra.run.smithi012.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T21:48:00.513 INFO:teuthology.orchestra.run.smithi012.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 423 kB/s | 35 kB 00:00 2024-04-26T21:48:00.580 INFO:teuthology.orchestra.run.smithi012.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-26T21:48:00.647 INFO:teuthology.orchestra.run.smithi012.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T21:48:00.764 INFO:teuthology.orchestra.run.smithi012.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T21:48:00.797 INFO:teuthology.orchestra.run.smithi012.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 516 kB/s | 77 kB 00:00 2024-04-26T21:48:00.814 INFO:teuthology.orchestra.run.smithi012.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 859 kB/s | 42 kB 00:00 2024-04-26T21:48:00.881 INFO:teuthology.orchestra.run.smithi012.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-26T21:48:00.915 INFO:teuthology.orchestra.run.smithi012.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-26T21:48:00.957 INFO:teuthology.orchestra.run.smithi012.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-26T21:48:00.982 INFO:teuthology.orchestra.run.smithi012.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 785 kB/s | 51 kB 00:00 2024-04-26T21:48:01.015 INFO:teuthology.orchestra.run.smithi012.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 931 kB/s | 54 kB 00:00 2024-04-26T21:48:01.133 INFO:teuthology.orchestra.run.smithi012.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-04-26T21:48:01.158 INFO:teuthology.orchestra.run.smithi012.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 295 kB/s | 42 kB 00:00 2024-04-26T21:48:01.200 INFO:teuthology.orchestra.run.smithi012.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 671 kB/s | 44 kB 00:00 2024-04-26T21:48:01.234 INFO:teuthology.orchestra.run.smithi012.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T21:48:01.259 INFO:teuthology.orchestra.run.smithi012.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 620 kB/s | 36 kB 00:00 2024-04-26T21:48:01.294 INFO:teuthology.orchestra.run.smithi012.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 820 kB/s | 47 kB 00:00 2024-04-26T21:48:01.327 INFO:teuthology.orchestra.run.smithi012.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T21:48:01.352 INFO:teuthology.orchestra.run.smithi012.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-26T21:48:01.386 INFO:teuthology.orchestra.run.smithi012.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T21:48:01.436 INFO:teuthology.orchestra.run.smithi012.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 768 kB/s | 64 kB 00:00 2024-04-26T21:48:01.461 INFO:teuthology.orchestra.run.smithi012.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-26T21:48:01.546 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T21:48:01.547 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T21:48:01.953 INFO:teuthology.orchestra.run.smithi012.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-26T21:48:02.178 INFO:teuthology.orchestra.run.smithi012.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-04-26T21:48:02.220 INFO:teuthology.orchestra.run.smithi012.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-26T21:48:02.278 INFO:teuthology.orchestra.run.smithi012.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 338 kB/s | 19 kB 00:00 2024-04-26T21:48:02.437 INFO:teuthology.orchestra.run.smithi012.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-26T21:48:02.754 INFO:teuthology.orchestra.run.smithi012.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-26T21:48:02.779 INFO:teuthology.orchestra.run.smithi012.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 76 kB/s | 26 kB 00:00 2024-04-26T21:48:02.821 INFO:teuthology.orchestra.run.smithi012.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 738 kB/s | 49 kB 00:00 2024-04-26T21:48:02.855 INFO:teuthology.orchestra.run.smithi012.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 905 kB/s | 67 kB 00:00 2024-04-26T21:48:02.897 INFO:teuthology.orchestra.run.smithi012.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T21:48:02.989 INFO:teuthology.orchestra.run.smithi012.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-26T21:48:03.031 INFO:teuthology.orchestra.run.smithi012.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-26T21:48:03.073 INFO:teuthology.orchestra.run.smithi012.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 524 kB/s | 43 kB 00:00 2024-04-26T21:48:03.107 INFO:teuthology.orchestra.run.smithi012.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-04-26T21:48:03.140 INFO:teuthology.orchestra.run.smithi012.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-04-26T21:48:03.165 INFO:teuthology.orchestra.run.smithi012.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-26T21:48:03.218 INFO:teuthology.orchestra.run.smithi012.stdout:(118/152): protobuf-compiler-3.14.0-13.el9.x86_ 492 kB/s | 863 kB 00:01 2024-04-26T21:48:03.251 INFO:teuthology.orchestra.run.smithi012.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T21:48:03.285 INFO:teuthology.orchestra.run.smithi012.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-26T21:48:03.341 INFO:teuthology.orchestra.run.smithi012.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-04-26T21:48:03.394 INFO:teuthology.orchestra.run.smithi012.stdout:(122/152): python3-colorama-0.4.4-12.el9.noarch 203 kB/s | 35 kB 00:00 2024-04-26T21:48:03.427 INFO:teuthology.orchestra.run.smithi012.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-26T21:48:03.477 INFO:teuthology.orchestra.run.smithi012.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-04-26T21:48:03.528 INFO:teuthology.orchestra.run.smithi012.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-04-26T21:48:03.578 INFO:teuthology.orchestra.run.smithi012.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-04-26T21:48:03.628 INFO:teuthology.orchestra.run.smithi012.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-04-26T21:48:03.679 INFO:teuthology.orchestra.run.smithi012.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-04-26T21:48:03.737 INFO:teuthology.orchestra.run.smithi012.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-26T21:48:03.787 INFO:teuthology.orchestra.run.smithi012.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 430 kB/s | 46 kB 00:00 2024-04-26T21:48:03.813 INFO:teuthology.orchestra.run.smithi012.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T21:48:03.888 INFO:teuthology.orchestra.run.smithi012.stdout:(132/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-04-26T21:48:03.938 INFO:teuthology.orchestra.run.smithi012.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 386 kB/s | 58 kB 00:00 2024-04-26T21:48:03.989 INFO:teuthology.orchestra.run.smithi012.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-26T21:48:04.022 INFO:teuthology.orchestra.run.smithi012.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-26T21:48:04.047 INFO:teuthology.orchestra.run.smithi012.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 833 kB/s | 90 kB 00:00 2024-04-26T21:48:04.098 INFO:teuthology.orchestra.run.smithi012.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 262 kB/s | 28 kB 00:00 2024-04-26T21:48:04.123 INFO:teuthology.orchestra.run.smithi012.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-26T21:48:04.156 INFO:teuthology.orchestra.run.smithi012.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-26T21:48:04.182 INFO:teuthology.orchestra.run.smithi012.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 717 kB/s | 59 kB 00:00 2024-04-26T21:48:04.207 INFO:teuthology.orchestra.run.smithi012.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-04-26T21:48:04.232 INFO:teuthology.orchestra.run.smithi012.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 686 kB/s | 51 kB 00:00 2024-04-26T21:48:04.266 INFO:teuthology.orchestra.run.smithi012.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-26T21:48:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 904 kB/s | 90 kB 00:00 2024-04-26T21:48:04.333 INFO:teuthology.orchestra.run.smithi012.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 303 kB/s | 20 kB 00:00 2024-04-26T21:48:04.373 INFO:teuthology.orchestra.run.smithi012.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-26T21:48:04.450 INFO:teuthology.orchestra.run.smithi012.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-26T21:48:04.475 INFO:teuthology.orchestra.run.smithi012.stdout:(148/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-26T21:48:04.684 INFO:teuthology.orchestra.run.smithi012.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-26T21:48:04.868 INFO:teuthology.orchestra.run.smithi012.stdout:(150/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-04-26T21:48:04.960 INFO:teuthology.orchestra.run.smithi012.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 3.4 MB 00:00 2024-04-26T21:48:05.068 INFO:teuthology.orchestra.run.smithi012.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.3 MB/s | 2.4 MB 00:00 2024-04-26T21:48:05.075 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:48:05.076 INFO:teuthology.orchestra.run.smithi012.stdout:Total 9.7 MB/s | 222 MB 00:22 2024-04-26T21:48:06.193 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-04-26T21:48:06.289 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:48:06.327 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:48:06.381 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:48:06.415 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:48:06.440 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:48:06.483 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:48:06.807 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:48:06.916 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:07.039 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:07.068 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:07.159 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:48:07.338 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:48:07.394 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:48:07.422 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:07.526 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:07.556 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:07.625 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T21:48:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:07.700 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:48:07.747 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:48:07.857 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:07.890 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:07.893 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:48:07.928 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-04-26T21:48:07.997 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:48:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:48:08.095 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:48:08.119 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:48:08.131 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:48:08.163 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:48:08.166 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:48:08.187 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-04-26T21:48:08.187 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-04-26T21:48:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:48:08.265 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:48:08.308 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:48:08.341 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:48:08.369 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:48:08.437 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:48:08.490 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:48:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:48:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:48:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:48:08.603 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:08.643 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:48:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:48:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:48:08.725 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:48:08.755 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:08.817 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:48:08.855 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:08.863 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:48:08.884 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:48:08.937 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:48:09.032 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:48:09.079 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:48:09.081 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:48:09.118 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:48:09.221 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:09.252 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:09.315 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:09.345 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:09.368 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:09.561 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:09.594 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:09.698 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:48:09.709 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:48:09.787 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:48:09.804 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:48:09.836 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:48:09.844 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:48:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:48:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:48:09.928 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:48:09.944 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:48:09.984 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:48:10.013 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:48:10.056 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:48:10.088 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:48:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:48:10.146 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:48:10.187 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:48:10.207 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:48:10.239 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:48:10.262 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:48:10.278 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:48:10.323 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:48:10.370 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:48:10.400 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:48:10.406 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:48:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:48:10.441 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:48:10.483 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:48:10.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:48:10.513 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:48:10.519 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:48:10.562 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:48:10.582 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:48:10.605 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:48:10.628 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:48:10.701 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:48:10.741 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:48:10.785 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:48:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:48:10.833 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:48:10.857 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:48:10.888 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:48:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:48:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:48:10.999 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-04-26T21:48:11.000 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-04-26T21:48:11.022 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:48:11.047 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:48:11.072 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:48:11.148 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:48:11.273 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:48:11.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:48:11.332 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:48:11.418 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:48:11.450 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:48:11.498 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:48:11.502 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:48:11.530 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:48:11.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:48:11.559 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:48:11.573 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:48:11.594 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:48:11.611 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:48:11.627 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:48:11.661 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:48:11.670 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:48:11.703 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:48:11.790 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:48:11.827 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:48:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:48:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:48:11.889 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:48:11.913 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:48:11.943 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:48:12.011 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:48:12.016 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:48:12.048 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:48:12.067 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:48:12.085 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:48:12.131 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:48:12.168 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:48:12.211 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:48:12.261 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:48:12.403 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:48:12.458 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:48:12.489 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:48:12.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:48:12.623 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:48:12.664 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:48:12.704 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:48:12.730 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:48:12.754 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:48:12.788 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:48:12.828 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:48:12.839 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:48:12.873 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:48:12.955 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:48:12.994 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:48:12.995 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:48:13.023 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:48:13.047 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:48:13.058 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:48:13.097 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:48:13.128 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:48:13.134 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:48:13.158 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:48:13.166 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:48:13.193 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:48:13.226 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:48:13.276 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:48:13.311 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:48:13.344 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:48:13.373 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:48:13.417 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:48:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:48:13.474 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:48:13.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:48:13.553 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:48:13.651 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:48:13.700 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:48:14.098 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:48:14.213 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:48:14.424 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:48:14.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:48:14.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:48:14.603 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:48:14.655 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:48:14.743 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:48:14.783 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:48:14.969 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:48:15.025 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:48:15.108 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:48:15.443 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:48:15.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:48:15.564 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:48:15.597 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:48:15.748 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:48:15.862 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:48:15.908 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:48:15.952 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:16.042 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:16.073 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:48:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:48:16.588 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:48:16.622 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:16.693 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:16.715 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:48:17.087 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:48:17.177 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:48:17.205 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:48:17.238 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:48:17.493 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-04-26T21:48:17.550 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:48:17.582 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:48:17.585 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:17.651 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:48:17.676 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:17.707 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:48:17.742 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:48:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:48:17.851 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:48:17.892 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:48:18.025 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:48:18.221 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:48:18.255 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:18.317 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:18.340 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:48:18.469 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:48:18.573 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:18.830 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:18.969 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:18.977 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:19.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:19.027 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:48:19.422 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:48:19.484 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:48:19.512 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:19.552 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:19.599 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:19.679 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:19.705 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:19.706 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:19.706 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:48:19.706 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:48:19.706 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:19.733 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:19.817 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:48:19.847 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:19.919 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:48:20.208 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:20.265 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:48:20.462 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:48:20.589 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:48:20.618 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:20.631 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:48:20.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:20.681 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:48:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:48:20.854 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:48:20.920 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:48:20.968 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:48:21.006 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:48:21.129 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:48:21.195 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:48:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:21.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:48:21.477 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:48:21.528 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:48:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:48:21.680 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:48:21.748 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:48:21.860 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:48:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:48:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:48:22.835 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:48:22.949 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:48:22.998 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:48:23.045 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:48:23.094 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:48:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:48:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:48:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:48:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:48:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:48:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:48:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:48:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:48:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:48:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:48:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:48:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:48:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:48:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:48:24.784 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:48:24.966 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:48:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:48:25.117 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:48:25.175 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:48:25.233 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:48:25.451 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:48:25.660 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:48:25.740 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:48:25.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:48:25.899 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:48:25.996 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:48:26.082 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:48:26.130 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:48:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:48:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:48:26.327 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:48:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:48:26.520 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:48:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:48:26.652 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:48:26.727 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:48:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:48:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:48:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:48:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:48:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:48:28.113 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:48:28.184 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:48:28.292 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:48:28.332 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:48:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:48:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:48:28.712 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:48:29.586 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:48:29.739 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-04-26T21:48:31.268 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:48:31.429 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:31.445 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:31.445 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:31.445 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:48:31.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:31.510 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:48:31.550 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:48:31.595 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:48:32.004 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:48:32.047 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:48:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:48:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:48:32.890 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:32.890 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-04-26T21:48:32.890 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-04-26T21:48:32.891 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:48:32.940 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:32.966 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:33.044 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:48:33.064 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:33.069 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:48:33.080 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:33.080 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:33.080 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:48:33.080 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:33.208 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:33.232 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:33.677 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:33.699 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:33.699 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:33.699 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:48:33.700 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:33.700 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:33.700 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:33.807 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:48:33.881 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:48:33.936 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:48:34.123 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:48:34.173 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:48:34.248 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:48:34.337 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:48:34.477 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:48:34.511 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:48:34.543 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:48:34.562 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:48:34.567 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:34.835 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:34.858 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:34.965 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:34.994 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:35.027 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:48:35.097 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:48:35.126 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:35.310 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:35.333 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:35.333 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:35.333 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:48:35.333 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:35.334 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:35.334 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:35.449 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:48:35.497 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:48:35.553 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:48:35.738 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:48:35.786 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:48:35.864 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:48:35.893 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:35.932 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:35.952 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:48:36.094 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:48:36.136 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:48:36.168 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:48:36.169 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:36.203 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:36.205 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:48:36.613 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:48:36.669 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:36.674 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:36.703 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:36.735 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:48:36.805 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:48:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:48:36.833 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:48:36.833 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:48:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:36.835 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:37.170 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:37.204 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:37.306 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:37.466 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:48:37.497 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:37.520 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:37.521 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:37.521 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:48:37.521 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:37.521 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:37.521 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:37.565 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:37.576 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:37.674 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:48:37.722 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:48:37.877 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:37.912 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:38.665 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:48:38.701 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:38.723 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:38.724 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:38.724 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:48:38.724 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:38.724 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:38.724 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:38.768 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:48:38.790 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:48:38.790 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:38.790 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:48:38.790 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:38.844 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:48:38.866 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:48:38.866 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:38.867 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:48:38.867 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:48:38.867 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:48:38.867 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:38.883 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:38.921 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:39.022 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:39.178 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:48:39.184 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:48:39.200 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:48:39.200 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:39.200 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:48:39.201 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:48:39.201 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:48:39.201 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:39.214 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:39.239 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:39.290 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:39.302 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:39.400 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:48:39.447 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:48:40.388 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:48:40.426 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:40.449 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:40.449 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:40.449 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:48:40.449 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:40.449 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:40.450 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:40.510 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:48:40.533 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:48:40.533 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:40.533 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:48:40.533 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:40.578 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:48:40.600 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:48:40.600 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:40.600 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:48:40.600 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:48:40.601 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:48:40.601 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:40.903 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:48:40.926 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:48:40.927 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:40.927 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:48:40.927 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:48:40.927 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:48:40.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:43.823 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:48:43.882 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:48:43.923 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:48:43.955 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:48:43.982 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:48:44.166 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:48:44.220 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:48:44.250 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:48:44.842 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:45.549 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:48:45.590 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:48:45.640 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:48:45.672 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:48:45.699 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:48:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:48:45.845 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:48:45.882 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:48:45.886 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:48:45.929 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:48:45.959 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:48:45.976 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:48:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:48:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:48:46.100 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:48:46.125 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:48:46.125 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:48:46.223 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:48:46.290 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:46.524 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:46.565 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:46.585 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:48:47.421 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:48:47.504 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:48:47.562 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:48:47.643 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:48:47.683 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:48:47.683 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:48:47.776 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:48:47.817 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:48:47.817 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:48:47.909 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:48:47.974 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:50.326 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-04-26T21:48:50.327 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:50.575 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:50.625 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:48:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:48:50.625 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:48:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:48:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:48:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:48:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:48:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:48:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:48:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:48:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:48:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:48:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:48:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:48:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:48:52.334 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:48:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:52.693 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:48:52.772 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:53.285 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:53.362 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:48:53.362 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:53.362 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:48:53.362 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:53.362 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:48:53.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:53.495 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:48:53.557 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:48:53.629 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:48:53.875 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:48:53.940 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:48:54.003 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:54.003 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:48:54.004 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:48:54.005 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:48:54.006 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:48:54.016 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:48:54.017 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:48:54.018 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:48:54.019 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:48:54.020 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:48:54.021 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:48:54.022 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:48:54.023 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:48:54.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:48:54.025 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:48:54.032 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:48:54.147 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:48:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:48:54.350 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:54.350 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:54.350 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-04-26T21:48:54.350 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.351 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:48:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.354 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:48:54.355 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:48:54.356 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:48:54.357 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:48:54.358 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:48:54.359 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:48:54.360 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:48:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:48:54.362 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:48:54.363 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:48:54.363 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:48:54.363 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-04-26T21:48:54.389 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:48:54.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:48:54.463 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:48:54.737 DEBUG:teuthology.parallel:result is None 2024-04-26T21:48:55.139 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:55.184 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:48:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:48:55.364 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:48:55.401 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:56.148 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:48:56.148 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:56.148 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.149 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:48:56.151 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:48:56.152 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:48:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:48:56.154 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:48:56.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:48:56.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:48:56.158 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:48:56.159 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T21:48:56.160 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T21:48:56.545 DEBUG:teuthology.parallel:result is None 2024-04-26T21:48:56.775 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:48:56.820 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:57.982 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:48:58.020 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:58.174 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:48:58.358 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:48:58.388 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:58.470 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:48:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:48:58.470 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:58.471 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:48:58.471 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:48:58.685 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:48:58.755 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:48:59.814 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:48:59.850 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:48:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:48:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:49:00.048 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:49:00.048 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:49:00.048 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:49:00.048 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:00.143 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:49:00.224 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:00.610 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:49:00.687 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:49:00.687 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:49:00.688 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:49:00.688 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:49:00.688 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:49:00.688 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:06.000 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:49:06.046 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:49:06.138 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:49:06.180 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:49:06.229 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:49:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:49:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:49:06.649 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:49:07.558 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:49:07.605 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:49:07.669 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:49:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:49:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:49:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:49:09.446 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:49:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:49:09.493 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:49:09.620 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:49:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:49:09.667 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:49:09.816 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:49:09.960 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:49:10.271 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:49:10.271 INFO:teuthology.orchestra.run.smithi012.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:49:10.271 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:16.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:49:16.135 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:49:16.136 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:49:16.137 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:49:16.138 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:49:16.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:49:16.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:49:16.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:49:16.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:49:16.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:49:16.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:49:16.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:49:16.159 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:49:16.160 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:49:16.161 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:49:16.162 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:49:16.163 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:49:16.164 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:49:16.165 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:49:16.165 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:49:16.165 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:49:18.525 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:49:18.526 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.527 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.528 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:49:18.529 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:49:18.530 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.531 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:49:18.532 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:49:18.533 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:49:18.534 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:49:18.535 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:49:18.536 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:49:18.537 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:49:18.538 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:49:18.539 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:49:18.540 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:49:18.541 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-04-26T21:49:19.259 DEBUG:teuthology.parallel:result is None 2024-04-26T21:49:19.259 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:19.479 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:49:19.542 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.542 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.542 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:49:19.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:19.756 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:49:19.782 INFO:teuthology.orchestra.run.smithi077.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.782 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.782 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:49:19.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:19.919 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:49:19.945 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.945 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:49:19.945 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:49:19.946 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:49:19.946 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:49:19.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:49:20.027 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:49:20.027 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:49:20.058 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:49:20.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:49:20.089 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:49:20.089 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:49:20.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:49:20.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:49:20.231 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:49:20.231 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:49:20.259 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:49:20.326 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:49:20.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:49:20.355 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:49:20.423 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:49:20.423 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:49:20.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:49:20.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:49:20.595 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:49:20.595 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:49:20.619 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:49:20.682 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:49:20.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:49:20.707 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:49:20.770 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:49:20.770 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:49:20.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:49:20.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:49:20.943 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:49:20.943 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:49:20.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:49:21.031 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:49:21.031 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:49:21.055 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:49:21.118 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T21:49:21.211 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T21:49:21.211 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:21.212 INFO:tasks.cephadm:Cluster fsid is d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:49:21.212 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T21:49:21.212 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.77', 'mon.c': '172.21.15.154'} 2024-04-26T21:49:21.213 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-04-26T21:49:21.213 INFO:tasks.cephadm:First mgr is x 2024-04-26T21:49:21.213 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T21:49:21.213 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-04-26T21:49:21.281 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-04-26T21:49:21.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-04-26T21:49:21.328 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T21:49:21.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:21.467 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T21:49:21.601 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:49:21.602 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T21:49:21.602 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:22.217 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:49 /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:22.217 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:22.808 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:49 /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:22.809 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:23.399 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:49 /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:23.400 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:23.451 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:23.469 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:49:23.503 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T21:49:23.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:49:23.510 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:49:23.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:49:25.063 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:49:25.104 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:49:25.594 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:50:00.435 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T21:50:00.435 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:50:00.436 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:50:00.436 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-04-26T21:50:00.436 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:50:00.436 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-04-26T21:50:00.436 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-04-26T21:50:09.866 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-26T21:50:14.187 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T21:50:14.188 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T21:50:14.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-04-26T21:50:14.293 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-04-26T21:50:14.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-04-26T21:50:14.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-04-26T21:50:14.435 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-04-26T21:50:14.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-04-26T21:50:14.490 INFO:tasks.cephadm:Writing seed config... 2024-04-26T21:50:14.491 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T21:50:14.491 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T21:50:14.492 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T21:50:14.493 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T21:50:14.494 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T21:50:14.495 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T21:50:14.495 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:50:14.495 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T21:50:14.549 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d6d929b6-0416-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T21:50:14.550 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T21:50:14.597 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T21:50:14.642 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T21:50:14.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:50:14.903 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:50:14.903 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'd6d929b6-0416-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-04-26T21:50:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T21:50:14.904 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-04-26T21:50:14.959 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:50:14.959 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-04-26T21:50:14.959 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-04-26T21:50:14.974 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:50:14.974 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:50:14.990 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:50:14.991 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:50:15.012 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-04-26T21:50:15.030 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-04-26T21:50:15.030 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:50:15.030 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-04-26T21:50:15.082 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:50:15.083 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T21:50:15.083 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-04-26T21:50:15.083 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-04-26T21:50:15.101 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:50:15.101 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:50:15.116 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:50:15.117 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:50:15.139 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-04-26T21:50:15.161 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-04-26T21:50:15.162 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:50:15.162 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-04-26T21:50:15.162 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:15.163 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 140521172024480 on /run/cephadm/d6d929b6-0416-11ef-bc93-c7b262605968.lock 2024-04-26T21:50:15.163 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 140521172024480 acquired on /run/cephadm/d6d929b6-0416-11ef-bc93-c7b262605968.lock 2024-04-26T21:50:15.165 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-04-26T21:50:15.165 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-04-26T21:50:15.166 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-04-26T21:50:15.180 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-04-26T21:50:15.180 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-04-26T21:50:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T21:50:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T21:50:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T21:50:15.204 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T21:50:15.204 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T21:50:15.205 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:50:15.205 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T21:50:15.205 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-04-26T21:50:15.205 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:50:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-04-26T21:50:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-04-26T21:50:15.208 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T21:50:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T21:50:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:50:15.882 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T21:50:17.046 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:50:17.046 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:50:17.047 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T21:50:18.199 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-04-26T21:50:18.199 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-04-26T21:50:19.297 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCaISxmta4RLBAACXXMrc3zEeha22xTtNJ5Sw== 2024-04-26T21:50:20.493 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCbISxmqxXPLxAASeFnHKngZQPFNXGOV0av6w== 2024-04-26T21:50:21.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCcISxmb19SORAAibGnbY+qxwRtmd28RPrmZg== 2024-04-26T21:50:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-04-26T21:50:22.873 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:50:22.874 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:50:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-04-26T21:50:23.921 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-04-26T21:50:24.323 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T21:50:24.574 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T21:50:24.876 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target. 2024-04-26T21:50:24.876 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target. 2024-04-26T21:50:25.263 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:50:25.263 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T21:50:25.566 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target.wants/ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968@.service. 2024-04-26T21:50:26.294 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 podman[27797]: 2024-04-26 21:50:26.019938466 +0000 UTC m=+0.154739807 container create 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T21:50:26.294 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 podman[27797]: 2024-04-26 21:50:26.291964649 +0000 UTC m=+0.426765997 container init 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:50:26.426 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:50:26.426 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-26T21:50:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:50:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:50:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-26T21:50:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:50:26.447 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-04-26T21:50:26.447 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-04-26T21:50:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 podman[27797]: 2024-04-26 21:50:26.302996491 +0000 UTC m=+0.437797805 container start 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T21:50:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 ceph-mon[27815]: mkfs d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 ceph-mon[27815]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:50:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 bash[27797]: 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 2024-04-26T21:50:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:26 smithi012 systemd[1]: Started Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:50:27.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:50:27.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: monmap epoch 1 2024-04-26T21:50:27.495 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: last_changed 2024-04-26T21:50:22.222336+0000 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: min_mon_release 19 (squid) 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: election_strategy: 1 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: fsmap 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: mgrmap e1: no daemons active 2024-04-26T21:50:27.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:27 smithi012 ceph-mon[27815]: from='client.? 172.21.15.12:0/3536655667' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T21:50:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T21:50:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T21:50:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-04-26T21:50:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.772643s) 2024-04-26T21:50:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T21:50:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T21:50:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:27.781 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-04-26T21:50:27.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T21:50:27.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T21:50:27.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T21:50:27.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T21:50:27.786 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.058+0000 7f223d400700 1 Processor -- start 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.058+0000 7f223d400700 1 -- start start 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.058+0000 7f223d400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.059+0000 7f223d400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2238083560 con 0x7f223810fce0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.059+0000 7f2236800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.060+0000 7f2236800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33522/0 (socket says 172.21.15.12:33522) 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.060+0000 7f2236800700 1 -- 172.21.15.12:0/292516855 learned_addr learned my addr 172.21.15.12:0/292516855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.060+0000 7f2236800700 1 -- 172.21.15.12:0/292516855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2238112610 con 0x7f223810fce0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f2236800700 1 --2- 172.21.15.12:0/292516855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22280098a0 tx=0x7f2228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ce6143e17161c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f2235e00700 1 -- 172.21.15.12:0/292516855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2228004160 con 0x7f223810fce0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f2235e00700 1 -- 172.21.15.12:0/292516855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2228004300 con 0x7f223810fce0 2024-04-26T21:50:27.787 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f2235e00700 1 -- 172.21.15.12:0/292516855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22280036a0 con 0x7f223810fce0 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f223d400700 1 -- 172.21.15.12:0/292516855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 msgr2=0x7f22381120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.061+0000 7f223d400700 1 --2- 172.21.15.12:0/292516855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22280098a0 tx=0x7f2228009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.062+0000 7f223d400700 1 -- 172.21.15.12:0/292516855 shutdown_connections 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.062+0000 7f223d400700 1 --2- 172.21.15.12:0/292516855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381120d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.062+0000 7f223d400700 1 -- 172.21.15.12:0/292516855 >> 172.21.15.12:0/292516855 conn(0x7f223807e2e0 msgr2=0x7f2238080700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.062+0000 7f223d400700 1 -- 172.21.15.12:0/292516855 shutdown_connections 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.062+0000 7f223d400700 1 -- 172.21.15.12:0/292516855 wait complete. 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.063+0000 7f223d400700 1 Processor -- start 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.063+0000 7f223d400700 1 -- start start 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.063+0000 7f223d400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.063+0000 7f223d400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2238112b40 con 0x7f223810fce0 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.064+0000 7f2236800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.064+0000 7f2236800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33524/0 (socket says 172.21.15.12:33524) 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.064+0000 7f2236800700 1 -- 172.21.15.12:0/3536655667 learned_addr learned my addr 172.21.15.12:0/3536655667 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.064+0000 7f2236800700 1 -- 172.21.15.12:0/3536655667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22381a2f40 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.064+0000 7f2236800700 1 --2- 172.21.15.12:0/3536655667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2228003db0 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2228011630 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22381a31d0 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2228011950 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22381a5ec0 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f222801a460 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.065+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f222801b780 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.066+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2238064a80 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.069+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f222801b980 con 0x7f223810fce0 2024-04-26T21:50:27.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.152+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f223802e6e0 con 0x7f223810fce0 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.153+0000 7f222f400700 1 -- 172.21.15.12:0/3536655667 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f222801bb60 con 0x7f223810fce0 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 msgr2=0x7f22381a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 --2- 172.21.15.12:0/3536655667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228004750 comp rx=0 tx=0).stop 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 shutdown_connections 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 --2- 172.21.15.12:0/3536655667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223810fce0 0x7f22381a2a00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 >> 172.21.15.12:0/3536655667 conn(0x7f223807e2e0 msgr2=0x7f22380805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.154+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 shutdown_connections 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:27.155+0000 7f223d400700 1 -- 172.21.15.12:0/3536655667 wait complete. 2024-04-26T21:50:27.790 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-04-26T21:50:27.791 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T21:50:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:28 smithi012 ceph-mon[27815]: from='client.? 172.21.15.12:0/2411963428' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:50:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:28 smithi012 ceph-mon[27815]: from='client.? 172.21.15.12:0/2411963428' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T21:50:29.168 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:29.169 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:50:29.170 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.416+0000 7f02c5400700 1 Processor -- start 2024-04-26T21:50:29.177 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.416+0000 7f02c5400700 1 -- start start 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.416+0000 7f02c5400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.417+0000 7f02c5400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02c010b800 con 0x7f02c010aea0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.417+0000 7f02be800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.417+0000 7f02be800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33528/0 (socket says 172.21.15.12:33528) 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.417+0000 7f02be800700 1 -- 172.21.15.12:0/3976805578 learned_addr learned my addr 172.21.15.12:0/3976805578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.418+0000 7f02be800700 1 -- 172.21.15.12:0/3976805578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c010c090 con 0x7f02c010aea0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.418+0000 7f02be800700 1 --2- 172.21.15.12:0/3976805578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5baa4bb0a431d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02bd400700 1 -- 172.21.15.12:0/3976805578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0004160 con 0x7f02c010aea0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02bd400700 1 -- 172.21.15.12:0/3976805578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0004300 con 0x7f02c010aea0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02bd400700 1 -- 172.21.15.12:0/3976805578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b00036a0 con 0x7f02c010aea0 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02c5400700 1 -- 172.21.15.12:0/3976805578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 msgr2=0x7f02c010b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02c5400700 1 --2- 172.21.15.12:0/3976805578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02c5400700 1 -- 172.21.15.12:0/3976805578 shutdown_connections 2024-04-26T21:50:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02c5400700 1 --2- 172.21.15.12:0/3976805578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c010b2c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.419+0000 7f02c5400700 1 -- 172.21.15.12:0/3976805578 >> 172.21.15.12:0/3976805578 conn(0x7f02c007e2e0 msgr2=0x7f02c007e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.420+0000 7f02c5400700 1 -- 172.21.15.12:0/3976805578 shutdown_connections 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.420+0000 7f02c5400700 1 -- 172.21.15.12:0/3976805578 wait complete. 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.421+0000 7f02c5400700 1 Processor -- start 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.421+0000 7f02c5400700 1 -- start start 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02c5400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02c5400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02c010c560 con 0x7f02c010aea0 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02be800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02be800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33530/0 (socket says 172.21.15.12:33530) 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02be800700 1 -- 172.21.15.12:0/2411963428 learned_addr learned my addr 172.21.15.12:0/2411963428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.422+0000 7f02be800700 1 -- 172.21.15.12:0/2411963428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c019eb60 con 0x7f02c010aea0 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02be800700 1 --2- 172.21.15.12:0/2411963428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f02b0000c00 tx=0x7f02b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0003db0 con 0x7f02c010aea0 2024-04-26T21:50:29.179 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0011630 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c019edf0 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c01a1ae0 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.423+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0011950 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.424+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02b001a460 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.424+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02b001b780 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.425+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c0064a80 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.427+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f02b0031020 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.506+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f02c019f2d0 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.513+0000 7f02b7400700 1 -- 172.21.15.12:0/2411963428 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f02c019f2d0 con 0x7f02c010aea0 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 msgr2=0x7f02c019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 --2- 172.21.15.12:0/2411963428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f02b0000c00 tx=0x7f02b0004750 comp rx=0 tx=0).stop 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 shutdown_connections 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 --2- 172.21.15.12:0/2411963428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02c010aea0 0x7f02c019e620 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.180 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 >> 172.21.15.12:0/2411963428 conn(0x7f02c007e2e0 msgr2=0x7f02c01951e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:29.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 shutdown_connections 2024-04-26T21:50:29.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:28.515+0000 7f02c5400700 1 -- 172.21.15.12:0/2411963428 wait complete. 2024-04-26T21:50:29.181 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-04-26T21:50:30.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:29 smithi012 ceph-mon[27815]: from='client.? 172.21.15.12:0/811195056' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.737+0000 7fc9f6a00700 1 Processor -- start 2024-04-26T21:50:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.737+0000 7fc9f6a00700 1 -- start start 2024-04-26T21:50:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.738+0000 7fc9f6a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.738+0000 7fc9f6a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0083560 con 0x7fc9f010fce0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.738+0000 7fc9efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.739+0000 7fc9efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44132/0 (socket says 172.21.15.12:44132) 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.739+0000 7fc9efe00700 1 -- 172.21.15.12:0/2279932039 learned_addr learned my addr 172.21.15.12:0/2279932039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.739+0000 7fc9efe00700 1 -- 172.21.15.12:0/2279932039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f0112610 con 0x7fc9f010fce0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.740+0000 7fc9efe00700 1 --2- 172.21.15.12:0/2279932039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098a0 tx=0x7fc9e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a70363c7031f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.740+0000 7fc9eea00700 1 -- 172.21.15.12:0/2279932039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0004160 con 0x7fc9f010fce0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.740+0000 7fc9eea00700 1 -- 172.21.15.12:0/2279932039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9e0004300 con 0x7fc9f010fce0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.740+0000 7fc9eea00700 1 -- 172.21.15.12:0/2279932039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0011af0 con 0x7fc9f010fce0 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 -- 172.21.15.12:0/2279932039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 msgr2=0x7fc9f01120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 --2- 172.21.15.12:0/2279932039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098a0 tx=0x7fc9e0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 -- 172.21.15.12:0/2279932039 shutdown_connections 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 --2- 172.21.15.12:0/2279932039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01120d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 -- 172.21.15.12:0/2279932039 >> 172.21.15.12:0/2279932039 conn(0x7fc9f007e2e0 msgr2=0x7fc9f0080700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 -- 172.21.15.12:0/2279932039 shutdown_connections 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.741+0000 7fc9f6a00700 1 -- 172.21.15.12:0/2279932039 wait complete. 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.742+0000 7fc9f6a00700 1 Processor -- start 2024-04-26T21:50:30.487 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.742+0000 7fc9f6a00700 1 -- start start 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9f6a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9f6a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0113690 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44140/0 (socket says 172.21.15.12:44140) 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9efe00700 1 -- 172.21.15.12:0/811195056 learned_addr learned my addr 172.21.15.12:0/811195056 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.743+0000 7fc9efe00700 1 -- 172.21.15.12:0/811195056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f01a37d0 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9efe00700 1 --2- 172.21.15.12:0/811195056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009fb0 tx=0x7fc9e0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e00036a0 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9e001a430 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01a3a60 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0019550 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.744+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a6750 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.745+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc9e0019b90 con 0x7fc9f010fce0 2024-04-26T21:50:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.745+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc9e0024730 con 0x7fc9f010fce0 2024-04-26T21:50:30.489 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.746+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f0064a80 con 0x7fc9f010fce0 2024-04-26T21:50:30.489 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.749+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc9e0043ec0 con 0x7fc9f010fce0 2024-04-26T21:50:30.489 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.829+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc9f01a6ca0 con 0x7fc9f010fce0 2024-04-26T21:50:30.489 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.830+0000 7fc9ecc00700 1 -- 172.21.15.12:0/811195056 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fc9e0032030 con 0x7fc9f010fce0 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 msgr2=0x7fc9f01a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 --2- 172.21.15.12:0/811195056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009fb0 tx=0x7fc9e0011940 comp rx=0 tx=0).stop 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 shutdown_connections 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 --2- 172.21.15.12:0/811195056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9f010fce0 0x7fc9f01a3290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 >> 172.21.15.12:0/811195056 conn(0x7fc9f007e2e0 msgr2=0x7fc9f007ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 shutdown_connections 2024-04-26T21:50:30.490 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:29.832+0000 7fc9f6a00700 1 -- 172.21.15.12:0/811195056 wait complete. 2024-04-26T21:50:30.491 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-04-26T21:50:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:30 smithi012 systemd[1]: Stopping Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:50:31.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a[27811]: 2024-04-26T21:50:30.972+0000 7fc271e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:50:31.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a[27811]: 2024-04-26T21:50:30.972+0000 7fc271e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T21:50:31.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:31 smithi012 podman[28060]: 2024-04-26 21:50:31.107173893 +0000 UTC m=+0.256741130 container died 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T21:50:31.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:31 smithi012 podman[28060]: 2024-04-26 21:50:31.514725636 +0000 UTC m=+0.664292896 container cleanup 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:31.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:31 smithi012 bash[28060]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:50:32.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:31 smithi012 podman[28074]: 2024-04-26 21:50:31.673954478 +0000 UTC m=+0.545680236 container remove 7361cd57f4a74f41fce6cc36e56aff9c51189bc85a1ccb9c319e85ba458d7063 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:50:32.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:32 smithi012 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T21:50:32.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:32 smithi012 systemd[1]: Stopped Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:50:32.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:32 smithi012 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.997s CPU time. 2024-04-26T21:50:32.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:32 smithi012 systemd[1]: Starting Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:50:33.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:32 smithi012 podman[28166]: 2024-04-26 21:50:32.479550024 +0000 UTC m=+0.036605615 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:36 smithi012 podman[28166]: 2024-04-26 21:50:36.665356482 +0000 UTC m=+4.222412105 container create 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T21:50:41.410 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T21:50:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 podman[28166]: 2024-04-26 21:50:41.309744575 +0000 UTC m=+8.866800205 container init 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T21:50:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 podman[28166]: 2024-04-26 21:50:41.321759963 +0000 UTC m=+8.878815570 container start 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T21:50:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:50:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: pidfile_write: ignore empty --pid-file 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: load: jerasure load: lrc 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Git sha 0 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: DB SUMMARY 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: DB Session ID: Y0WTOOGOOOW3PUM69674 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103970 ; 2024-04-26T21:50:41.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.env: 0x5632f0c43cc0 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.info_log: 0x5633022c7dc0 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.statistics: (nil) 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.use_fsync: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.db_log_dir: 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.wal_dir: 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:50:41.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.write_buffer_manager: 0x5633022990e0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.unordered_write: 0 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:50:41.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.row_cache: None 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.wal_filter: None 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.wal_compression: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:50:41.580 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:50:41.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_open_files: -1 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Compression algorithms supported: 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kZSTD supported: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:50:41.582 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:50:41.583 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:50:41.583 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.merge_operator: 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_filter: None 2024-04-26T21:50:41.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5633022c7f80) 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-04-26T21:50:41.585 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x5633022cb1f0 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-04-26T21:50:41.586 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:50:41.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression: NoCompression 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.num_levels: 7 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:50:41.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:50:41.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:50:41.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:50:41.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.ttl: 2592000 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:50:41.592 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:50:41.593 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b5f29275-08af-47e0-8f65-5f8508cedb38 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168241381819, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168241384884, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97163, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13897, "raw_average_key_size": 49, "raw_value_size": 89558, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714168241, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b5f29275-08af-47e0-8f65-5f8508cedb38", "db_session_id": "Y0WTOOGOOOW3PUM69674", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168241385113, "job": 1, "event": "recovery_finished"} 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5633023e8000 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: DB pointer 0x5633023d4000 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:50:41.594 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 98.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 98.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T21:50:41.595 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 51.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:50:41.596 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 9.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 9.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x5633022cb1f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???) e1 preinit fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).mds e1 new map 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).mds e1 print_map 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:50:41.597 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 bash[28166]: 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 systemd[1]: Started Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: monmap epoch 1 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: last_changed 2024-04-26T21:50:22.222336+0000 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:50:41.598 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: min_mon_release 19 (squid) 2024-04-26T21:50:41.599 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: election_strategy: 1 2024-04-26T21:50:41.599 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:50:41.599 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: fsmap 2024-04-26T21:50:41.599 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:50:41.599 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:41 smithi012 ceph-mon[28184]: mgrmap e1: no daemons active 2024-04-26T21:50:42.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.038+0000 7f8cab600700 1 Processor -- start 2024-04-26T21:50:42.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.038+0000 7f8cab600700 1 -- start start 2024-04-26T21:50:42.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.038+0000 7f8cab600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.038+0000 7f8cab600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410b7e0 con 0x7f8ca410ae80 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.039+0000 7f8ca8c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.039+0000 7f8ca8c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48936/0 (socket says 172.21.15.12:48936) 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.040+0000 7f8ca8c00700 1 -- 172.21.15.12:0/444615079 learned_addr learned my addr 172.21.15.12:0/444615079 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.040+0000 7f8ca8c00700 1 -- 172.21.15.12:0/444615079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca410c070 con 0x7f8ca410ae80 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.041+0000 7f8ca8c00700 1 --2- 172.21.15.12:0/444615079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c940098a0 tx=0x7f8c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f962143646b58d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.041+0000 7f8ca3e00700 1 -- 172.21.15.12:0/444615079 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94004160 con 0x7f8ca410ae80 2024-04-26T21:50:42.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.041+0000 7f8ca3e00700 1 -- 172.21.15.12:0/444615079 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c94004300 con 0x7f8ca410ae80 2024-04-26T21:50:42.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.041+0000 7f8ca3e00700 1 -- 172.21.15.12:0/444615079 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c94011af0 con 0x7f8ca410ae80 2024-04-26T21:50:42.713 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 -- 172.21.15.12:0/444615079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 msgr2=0x7f8ca410b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.714 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 --2- 172.21.15.12:0/444615079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c940098a0 tx=0x7f8c94009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.714 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 -- 172.21.15.12:0/444615079 shutdown_connections 2024-04-26T21:50:42.714 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 --2- 172.21.15.12:0/444615079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca410b2a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.714 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 -- 172.21.15.12:0/444615079 >> 172.21.15.12:0/444615079 conn(0x7f8ca407e2b0 msgr2=0x7f8ca407e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:42.714 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 -- 172.21.15.12:0/444615079 shutdown_connections 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.042+0000 7f8cab600700 1 -- 172.21.15.12:0/444615079 wait complete. 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.043+0000 7f8cab600700 1 Processor -- start 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.043+0000 7f8cab600700 1 -- start start 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8cab600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8cab600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410f330 con 0x7f8ca410ae80 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8ca8c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8ca8c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48938/0 (socket says 172.21.15.12:48938) 2024-04-26T21:50:42.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8ca8c00700 1 -- 172.21.15.12:0/2572032506 learned_addr learned my addr 172.21.15.12:0/2572032506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:42.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.044+0000 7f8ca8c00700 1 -- 172.21.15.12:0/2572032506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca419f340 con 0x7f8ca410ae80 2024-04-26T21:50:42.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8ca8c00700 1 --2- 172.21.15.12:0/2572032506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c94003f40 tx=0x7f8c94011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:42.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c940038b0 con 0x7f8ca410ae80 2024-04-26T21:50:42.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca419f5d0 con 0x7f8ca410ae80 2024-04-26T21:50:42.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c94003a50 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9401a5c0 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a22c0 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.045+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c9401ac50 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.046+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c94019700 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.046+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca4064a80 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.050+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8c94019900 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.128+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8ca41a2490 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.132+0000 7f8ca2000700 1 -- 172.21.15.12:0/2572032506 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f8c94019ae0 con 0x7f8ca410ae80 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.133+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 msgr2=0x7f8ca419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.133+0000 7f8cab600700 1 --2- 172.21.15.12:0/2572032506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c94003f40 tx=0x7f8c94011890 comp rx=0 tx=0).stop 2024-04-26T21:50:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.133+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 shutdown_connections 2024-04-26T21:50:42.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.133+0000 7f8cab600700 1 --2- 172.21.15.12:0/2572032506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca410ae80 0x7f8ca419ee00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:42.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.133+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 >> 172.21.15.12:0/2572032506 conn(0x7f8ca407e2b0 msgr2=0x7f8ca4108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:42.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.134+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 shutdown_connections 2024-04-26T21:50:42.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:42.134+0000 7f8cab600700 1 -- 172.21.15.12:0/2572032506 wait complete. 2024-04-26T21:50:42.719 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T21:50:42.747 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:50:42.747 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-04-26T21:50:42.749 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T21:50:42.750 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T21:50:43.179 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:50:43.180 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T21:50:43.446 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968.target.wants/ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d6d929b6-0416-11ef-bc93-c7b262605968@.service. 2024-04-26T21:50:43.479 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:43 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2572032506' entity='client.admin' 2024-04-26T21:50:43.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:43 smithi012 systemd[1]: Starting Ceph mgr.x for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:50:44.092 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:43 smithi012 podman[28408]: 2024-04-26 21:50:43.741252401 +0000 UTC m=+0.036511247 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:50:44.092 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:43 smithi012 podman[28408]: 2024-04-26 21:50:43.849950948 +0000 UTC m=+0.145209787 container create 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T21:50:44.093 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 podman[28408]: 2024-04-26 21:50:44.08107631 +0000 UTC m=+0.376335196 container init 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:44.093 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 podman[28408]: 2024-04-26 21:50:44.092083293 +0000 UTC m=+0.387342141 container start 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:50:44.249 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:50:44.249 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-26T21:50:44.270 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:50:44.271 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:50:44.271 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-26T21:50:44.271 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:50:44.295 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:50:44.295 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-26T21:50:44.317 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:50:44.317 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:50:44.317 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-26T21:50:44.318 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T21:50:44.318 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-04-26T21:50:44.319 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-04-26T21:50:44.471 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 bash[28408]: 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a 2024-04-26T21:50:44.471 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 systemd[1]: Started Ceph mgr.x for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:50:44.471 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:44.328+0000 7fd684fd5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:50:44.725 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:44.587+0000 7fd684fd5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:50:44.725 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:44.724+0000 7fd684fd5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:50:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:45 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/974133284' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:45.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:45.149+0000 7fd684fd5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:50:45.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:45.264+0000 7fd684fd5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:50:45.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:50:45.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:50:45.725 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:50:45.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:50:45.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:50:26.382820+0000", 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.929+0000 7f61ad200700 1 Processor -- start 2024-04-26T21:50:45.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.929+0000 7f61ad200700 1 -- start start 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.930+0000 7f61ad200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.930+0000 7f61ad200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61a807d8c0 con 0x7f61a807ee90 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.931+0000 7f61a7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.931+0000 7f61a7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48960/0 (socket says 172.21.15.12:48960) 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.931+0000 7f61a7400700 1 -- 172.21.15.12:0/339314370 learned_addr learned my addr 172.21.15.12:0/339314370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.931+0000 7f61a7400700 1 -- 172.21.15.12:0/339314370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a807da40 con 0x7f61a807ee90 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.931+0000 7f61a7400700 1 --2- 172.21.15.12:0/339314370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61980098a0 tx=0x7f6198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ef43d250e01497 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61a6000700 1 -- 172.21.15.12:0/339314370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6198004160 con 0x7f61a807ee90 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61a6000700 1 -- 172.21.15.12:0/339314370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6198004300 con 0x7f61a807ee90 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61a6000700 1 -- 172.21.15.12:0/339314370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6198011af0 con 0x7f61a807ee90 2024-04-26T21:50:45.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61ad200700 1 -- 172.21.15.12:0/339314370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 msgr2=0x7f61a807d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61ad200700 1 --2- 172.21.15.12:0/339314370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61980098a0 tx=0x7f6198009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61ad200700 1 -- 172.21.15.12:0/339314370 shutdown_connections 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61ad200700 1 --2- 172.21.15.12:0/339314370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a807d2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.932+0000 7f61ad200700 1 -- 172.21.15.12:0/339314370 >> 172.21.15.12:0/339314370 conn(0x7f61a8104050 msgr2=0x7f61a8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.933+0000 7f61ad200700 1 -- 172.21.15.12:0/339314370 shutdown_connections 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.933+0000 7f61ad200700 1 -- 172.21.15.12:0/339314370 wait complete. 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.934+0000 7f61ad200700 1 Processor -- start 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.934+0000 7f61ad200700 1 -- start start 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.934+0000 7f61ad200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.935+0000 7f61ad200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810f080 con 0x7f61a807ee90 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.935+0000 7f61a7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.935+0000 7f61a7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48974/0 (socket says 172.21.15.12:48974) 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.935+0000 7f61a7400700 1 -- 172.21.15.12:0/974133284 learned_addr learned my addr 172.21.15.12:0/974133284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:45.734 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.935+0000 7f61a7400700 1 -- 172.21.15.12:0/974133284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a819f150 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f61a7400700 1 --2- 172.21.15.12:0/974133284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6198003f40 tx=0x7f6198011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61980038b0 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6198003a50 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a819f3e0 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619801a5c0 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81a20d0 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f619801ac50 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.936+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6198019700 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.937+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a8064a80 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:44.940+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f61980199a0 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.021+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f61a81a2610 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.021+0000 7f619fe00700 1 -- 172.21.15.12:0/974133284 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6198019b80 con 0x7f61a807ee90 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 msgr2=0x7f61a819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:45.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 --2- 172.21.15.12:0/974133284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6198003f40 tx=0x7f6198011890 comp rx=0 tx=0).stop 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 shutdown_connections 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 --2- 172.21.15.12:0/974133284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a807ee90 0x7f61a819ec10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 >> 172.21.15.12:0/974133284 conn(0x7f61a8104050 msgr2=0x7f61a8104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 shutdown_connections 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:45.024+0000 7f61ad200700 1 -- 172.21.15.12:0/974133284 wait complete. 2024-04-26T21:50:45.736 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-04-26T21:50:47.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:46.933+0000 7fd684fd5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:50:47.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:47.164+0000 7fd684fd5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:50:47.724 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:47.387+0000 7fd684fd5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:50:48.040 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:47.781+0000 7fd684fd5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:50:48.040 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:47.919+0000 7fd684fd5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:50:48.294 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:48.180+0000 7fd684fd5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:50:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:48 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2764164272' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:48.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:48.450+0000 7fd684fd5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:50:49.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:50:49.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:50:49.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:50:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:50:49.131 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:50:26.382820+0000", 2024-04-26T21:50:49.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.342+0000 7f0f6f800700 1 Processor -- start 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.342+0000 7f0f6f800700 1 -- start start 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.343+0000 7f0f6f800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.343+0000 7f0f6f800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f681096b0 con 0x7f0f68108cc0 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.343+0000 7f0f6ce00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.343+0000 7f0f6ce00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48982/0 (socket says 172.21.15.12:48982) 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.343+0000 7f0f6ce00700 1 -- 172.21.15.12:0/1454655824 learned_addr learned my addr 172.21.15.12:0/1454655824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.344+0000 7f0f6ce00700 1 -- 172.21.15.12:0/1454655824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f68109ec0 con 0x7f0f68108cc0 2024-04-26T21:50:49.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.344+0000 7f0f6ce00700 1 --2- 172.21.15.12:0/1454655824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098a0 tx=0x7f0f5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d8786a6e33af6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.344+0000 7f0f67400700 1 -- 172.21.15.12:0/1454655824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c004160 con 0x7f0f68108cc0 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.344+0000 7f0f67400700 1 -- 172.21.15.12:0/1454655824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f5c004300 con 0x7f0f68108cc0 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.344+0000 7f0f67400700 1 -- 172.21.15.12:0/1454655824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c011af0 con 0x7f0f68108cc0 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 -- 172.21.15.12:0/1454655824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 msgr2=0x7f0f681090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 --2- 172.21.15.12:0/1454655824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098a0 tx=0x7f0f5c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 -- 172.21.15.12:0/1454655824 shutdown_connections 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 --2- 172.21.15.12:0/1454655824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f681090e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 -- 172.21.15.12:0/1454655824 >> 172.21.15.12:0/1454655824 conn(0x7f0f681042a0 msgr2=0x7f0f681066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 -- 172.21.15.12:0/1454655824 shutdown_connections 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.345+0000 7f0f6f800700 1 -- 172.21.15.12:0/1454655824 wait complete. 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.346+0000 7f0f6f800700 1 Processor -- start 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6f800700 1 -- start start 2024-04-26T21:50:49.134 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6f800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6f800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6810d270 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6ce00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6ce00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48994/0 (socket says 172.21.15.12:48994) 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6ce00700 1 -- 172.21.15.12:0/2764164272 learned_addr learned my addr 172.21.15.12:0/2764164272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.347+0000 7f0f6ce00700 1 -- 172.21.15.12:0/2764164272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6819f360 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f6ce00700 1 --2- 172.21.15.12:0/2764164272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000c00 tx=0x7f0f5c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c003b20 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f5c01a8c0 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6819f5f0 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c024940 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.348+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f681a22e0 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.349+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f5c022020 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.349+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f5c0193c0 con 0x7f0f68108cc0 2024-04-26T21:50:49.135 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.349+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f68064a80 con 0x7f0f68108cc0 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.352+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0f5c0042c0 con 0x7f0f68108cc0 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.430+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0f681a2820 con 0x7f0f68108cc0 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.431+0000 7f0f65600700 1 -- 172.21.15.12:0/2764164272 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0f5c02dde0 con 0x7f0f68108cc0 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.432+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 msgr2=0x7f0f6819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.432+0000 7f0f6f800700 1 --2- 172.21.15.12:0/2764164272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000c00 tx=0x7f0f5c011b70 comp rx=0 tx=0).stop 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.433+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 shutdown_connections 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.433+0000 7f0f6f800700 1 --2- 172.21.15.12:0/2764164272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f68108cc0 0x7f0f6819ee20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.433+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 >> 172.21.15.12:0/2764164272 conn(0x7f0f681042a0 msgr2=0x7f0f68104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.433+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 shutdown_connections 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:48.433+0000 7f0f6f800700 1 -- 172.21.15.12:0/2764164272 wait complete. 2024-04-26T21:50:49.136 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-04-26T21:50:49.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:49.291+0000 7fd684fd5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:50:50.261 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:49.857+0000 7fd684fd5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:50:50.261 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:49.990+0000 7fd684fd5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:50:50.262 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:50.261+0000 7fd684fd5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:50:51.277 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:50.925+0000 7fd684fd5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:50:51.277 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:51.052+0000 7fd684fd5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:50:51.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:51.387+0000 7fd684fd5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:50:51.834 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:51.575+0000 7fd684fd5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:50:51.834 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:51.711+0000 7fd684fd5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:50:52.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/384599625' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:52.139 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:51.871+0000 7fd684fd5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:50:52.428 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:52.251+0000 7fd684fd5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:50:52.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:50:52.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:50:26.382820+0000", 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.744+0000 7fadd9200700 1 Processor -- start 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.744+0000 7fadd9200700 1 -- start start 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.744+0000 7fadd9200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.745+0000 7fadd9200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41096b0 con 0x7fadd4108cc0 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.745+0000 7fadd1e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.745+0000 7fadd1e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60136/0 (socket says 172.21.15.12:60136) 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.745+0000 7fadd1e00700 1 -- 172.21.15.12:0/1485878943 learned_addr learned my addr 172.21.15.12:0/1485878943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.745+0000 7fadd1e00700 1 -- 172.21.15.12:0/1485878943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd4109ec0 con 0x7fadd4108cc0 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.746+0000 7fadd1e00700 1 --2- 172.21.15.12:0/1485878943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009e80 tx=0x7fadc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ac37e29c6258ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.746+0000 7fadd0a00700 1 -- 172.21.15.12:0/1485878943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4017070 con 0x7fadd4108cc0 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.746+0000 7fadd0a00700 1 -- 172.21.15.12:0/1485878943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fadc400b8c0 con 0x7fadd4108cc0 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.746+0000 7fadd0a00700 1 -- 172.21.15.12:0/1485878943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4004490 con 0x7fadd4108cc0 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 -- 172.21.15.12:0/1485878943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 msgr2=0x7fadd41090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 --2- 172.21.15.12:0/1485878943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009e80 tx=0x7fadc400b330 comp rx=0 tx=0).stop 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 -- 172.21.15.12:0/1485878943 shutdown_connections 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 --2- 172.21.15.12:0/1485878943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd41090e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 -- 172.21.15.12:0/1485878943 >> 172.21.15.12:0/1485878943 conn(0x7fadd41042a0 msgr2=0x7fadd41066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 -- 172.21.15.12:0/1485878943 shutdown_connections 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.747+0000 7fadd9200700 1 -- 172.21.15.12:0/1485878943 wait complete. 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.748+0000 7fadd9200700 1 Processor -- start 2024-04-26T21:50:52.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.748+0000 7fadd9200700 1 -- start start 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.748+0000 7fadd9200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.748+0000 7fadd9200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fadd410d270 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.749+0000 7fadd1e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.749+0000 7fadd1e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60144/0 (socket says 172.21.15.12:60144) 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.749+0000 7fadd1e00700 1 -- 172.21.15.12:0/384599625 learned_addr learned my addr 172.21.15.12:0/384599625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.749+0000 7fadd1e00700 1 -- 172.21.15.12:0/384599625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd407f0c0 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.749+0000 7fadd1e00700 1 --2- 172.21.15.12:0/384599625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadc4004a60 tx=0x7fadc400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.750+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4020020 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.750+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fadc4003c60 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.750+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4017040 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.750+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd407d2b0 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.750+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd407d7f0 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.751+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fadc4023020 con 0x7fadd4108cc0 2024-04-26T21:50:52.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.751+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fadc400f050 con 0x7fadd4108cc0 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.751+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4064a80 con 0x7fadd4108cc0 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.754+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fadc402a020 con 0x7fadd4108cc0 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.833+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fadd407dae0 con 0x7fadd4108cc0 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.834+0000 7fadcaa00700 1 -- 172.21.15.12:0/384599625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fadc401c5a0 con 0x7fadd4108cc0 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 msgr2=0x7fadd407eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 --2- 172.21.15.12:0/384599625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadc4004a60 tx=0x7fadc400bd50 comp rx=0 tx=0).stop 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 shutdown_connections 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 --2- 172.21.15.12:0/384599625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadd4108cc0 0x7fadd407eb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 >> 172.21.15.12:0/384599625 conn(0x7fadd41042a0 msgr2=0x7fadd4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 shutdown_connections 2024-04-26T21:50:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:51.836+0000 7fadd9200700 1 -- 172.21.15.12:0/384599625 wait complete. 2024-04-26T21:50:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-04-26T21:50:53.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:53.044+0000 7fd684fd5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:50:53.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:53.335+0000 7fd684fd5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: Activating manager daemon x 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: mgrmap e2: x(active, starting, since 0.00622171s) 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: Manager daemon x is now available 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' 2024-04-26T21:50:53.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:53 smithi012 ceph-mon[28184]: from='mgr.14100 172.21.15.12:0/2343257466' entity='mgr.x' 2024-04-26T21:50:55.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:55 smithi012 ceph-mon[28184]: mgrmap e3: x(active, since 1.02281s) 2024-04-26T21:50:56.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:56.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:50:56.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T21:50:56.109 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:50:56.110 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:50:56.111 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:50:56.120 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:50:56.121 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:50:26.382820+0000", 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.067+0000 7ff83a400700 1 Processor -- start 2024-04-26T21:50:56.122 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.068+0000 7ff83a400700 1 -- start start 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.068+0000 7ff83a400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.068+0000 7ff83a400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410b6f0 con 0x7ff83410ad00 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.068+0000 7ff839000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.069+0000 7ff839000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60224/0 (socket says 172.21.15.12:60224) 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.069+0000 7ff839000700 1 -- 172.21.15.12:0/53214895 learned_addr learned my addr 172.21.15.12:0/53214895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.069+0000 7ff839000700 1 -- 172.21.15.12:0/53214895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83410bf00 con 0x7ff83410ad00 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.069+0000 7ff839000700 1 --2- 172.21.15.12:0/53214895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8280098a0 tx=0x7ff828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a8bc7273387825 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.069+0000 7ff833400700 1 -- 172.21.15.12:0/53214895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828004160 con 0x7ff83410ad00 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff833400700 1 -- 172.21.15.12:0/53214895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff828004300 con 0x7ff83410ad00 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff833400700 1 -- 172.21.15.12:0/53214895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828011af0 con 0x7ff83410ad00 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 -- 172.21.15.12:0/53214895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 msgr2=0x7ff83410b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 --2- 172.21.15.12:0/53214895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8280098a0 tx=0x7ff828009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 -- 172.21.15.12:0/53214895 shutdown_connections 2024-04-26T21:50:56.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 --2- 172.21.15.12:0/53214895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83410b120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 -- 172.21.15.12:0/53214895 >> 172.21.15.12:0/53214895 conn(0x7ff83407e0e0 msgr2=0x7ff83407e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.070+0000 7ff83a400700 1 -- 172.21.15.12:0/53214895 shutdown_connections 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.071+0000 7ff83a400700 1 -- 172.21.15.12:0/53214895 wait complete. 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.071+0000 7ff83a400700 1 Processor -- start 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff83a400700 1 -- start start 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff83a400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff83a400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff83410f2b0 con 0x7ff83410ad00 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff839000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff839000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60238/0 (socket says 172.21.15.12:60238) 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.072+0000 7ff839000700 1 -- 172.21.15.12:0/2847556109 learned_addr learned my addr 172.21.15.12:0/2847556109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff839000700 1 -- 172.21.15.12:0/2847556109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83419f1f0 con 0x7ff83410ad00 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff839000700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff828000c00 tx=0x7ff828011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828003b20 con 0x7ff83410ad00 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff82801a8c0 con 0x7ff83410ad00 2024-04-26T21:50:56.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828024940 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.073+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83419f480 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.074+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8341a2170 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.075+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7ff828022020 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.075+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff834064a80 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.075+0000 7ff831600700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 0x7ff81c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.075+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff82804d240 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.076+0000 7ff833e00700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 0x7ff81c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.076+0000 7ff833e00700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 0x7ff81c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff820006fd0 tx=0x7ff820006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.082+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff82800be50 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.360+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff82801aa70 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.397+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff8341a2340 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.398+0000 7ff831600700 1 -- 172.21.15.12:0/2847556109 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff8341a2340 con 0x7ff83410ad00 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 msgr2=0x7ff81c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 0x7ff81c03b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff820006fd0 tx=0x7ff820006e40 comp rx=0 tx=0).stop 2024-04-26T21:50:56.125 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 msgr2=0x7ff83419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff828000c00 tx=0x7ff828011b70 comp rx=0 tx=0).stop 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 shutdown_connections 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7ff81c039490 0x7ff81c03b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 --2- 172.21.15.12:0/2847556109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff83410ad00 0x7ff83419ecb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 >> 172.21.15.12:0/2847556109 conn(0x7ff83407e0e0 msgr2=0x7ff834108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 shutdown_connections 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:55.402+0000 7ff83a400700 1 -- 172.21.15.12:0/2847556109 wait complete. 2024-04-26T21:50:56.126 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-04-26T21:50:56.636 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:56 smithi012 ceph-mon[28184]: mgrmap e4: x(active, since 2s) 2024-04-26T21:50:56.636 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:56 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2847556109' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:57.681 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:57 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2317951438' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:50:57.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:57.697 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:50:57.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:50:57.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:50:57.698 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:50:57.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:50:57.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:50:57.699 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:50:57.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:57.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:50:57.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:50:57.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-26T21:50:57.700 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:50:57.701 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:50:57.701 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:50:57.701 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:50:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:50:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:50:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.717+0000 7f59c1200700 1 Processor -- start 2024-04-26T21:50:57.702 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.718+0000 7f59c1200700 1 -- start start 2024-04-26T21:50:57.703 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.718+0000 7f59c1200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.703 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.718+0000 7f59c1200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc07d920 con 0x7f59bc07eef0 2024-04-26T21:50:57.703 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.719+0000 7f59b9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.704 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.719+0000 7f59b9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60242/0 (socket says 172.21.15.12:60242) 2024-04-26T21:50:57.704 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.719+0000 7f59b9e00700 1 -- 172.21.15.12:0/1152943955 learned_addr learned my addr 172.21.15.12:0/1152943955 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:57.704 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.720+0000 7f59b9e00700 1 -- 172.21.15.12:0/1152943955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc07daa0 con 0x7f59bc07eef0 2024-04-26T21:50:57.704 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.720+0000 7f59b9e00700 1 --2- 172.21.15.12:0/1152943955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098a0 tx=0x7f59ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3861cfee5b1944 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.705 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.720+0000 7f59b8a00700 1 -- 172.21.15.12:0/1152943955 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac004160 con 0x7f59bc07eef0 2024-04-26T21:50:57.705 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.720+0000 7f59b8a00700 1 -- 172.21.15.12:0/1152943955 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59ac004300 con 0x7f59bc07eef0 2024-04-26T21:50:57.705 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.720+0000 7f59b8a00700 1 -- 172.21.15.12:0/1152943955 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac011af0 con 0x7f59bc07eef0 2024-04-26T21:50:57.705 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 -- 172.21.15.12:0/1152943955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 msgr2=0x7f59bc07d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 --2- 172.21.15.12:0/1152943955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098a0 tx=0x7f59ac009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 -- 172.21.15.12:0/1152943955 shutdown_connections 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 --2- 172.21.15.12:0/1152943955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc07d350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 -- 172.21.15.12:0/1152943955 >> 172.21.15.12:0/1152943955 conn(0x7f59bc1042a0 msgr2=0x7f59bc1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 -- 172.21.15.12:0/1152943955 shutdown_connections 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.721+0000 7f59c1200700 1 -- 172.21.15.12:0/1152943955 wait complete. 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.722+0000 7f59c1200700 1 Processor -- start 2024-04-26T21:50:57.706 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.722+0000 7f59c1200700 1 -- start start 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59c1200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59c1200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10f280 con 0x7f59bc07eef0 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59b9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59b9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60258/0 (socket says 172.21.15.12:60258) 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59b9e00700 1 -- 172.21.15.12:0/2317951438 learned_addr learned my addr 172.21.15.12:0/2317951438 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.723+0000 7f59b9e00700 1 -- 172.21.15.12:0/2317951438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc19f350 con 0x7f59bc07eef0 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59b9e00700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000c00 tx=0x7f59ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.708 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac003b20 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59ac01a8c0 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc19f5e0 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac024940 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.724+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a22d0 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.726+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f59ac022020 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.726+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc064a80 con 0x7f59bc07eef0 2024-04-26T21:50:57.709 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.726+0000 7f59b2a00700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 0x7f59a003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.726+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59ac04d420 con 0x7f59bc07eef0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.726+0000 7f59b9400700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 0x7f59a003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.727+0000 7f59b9400700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 0x7f59a003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.732+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59ac04d180 con 0x7f59bc07eef0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.959+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f59bc1a24a0 con 0x7f59bc07eef0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.962+0000 7f59b2a00700 1 -- 172.21.15.12:0/2317951438 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f59bc1a24a0 con 0x7f59bc07eef0 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.965+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 msgr2=0x7f59a003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.965+0000 7f59c1200700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 0x7f59a003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8006e40 comp rx=0 tx=0).stop 2024-04-26T21:50:57.711 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.965+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 msgr2=0x7f59bc19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.965+0000 7f59c1200700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000c00 tx=0x7f59ac011b70 comp rx=0 tx=0).stop 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 shutdown_connections 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f59a00395b0 0x7f59a003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 --2- 172.21.15.12:0/2317951438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59bc07eef0 0x7f59bc19ee10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 >> 172.21.15.12:0/2317951438 conn(0x7f59bc1042a0 msgr2=0x7f59bc104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 shutdown_connections 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:56.966+0000 7f59c1200700 1 -- 172.21.15.12:0/2317951438 wait complete. 2024-04-26T21:50:57.712 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-04-26T21:50:58.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:58 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3171583166' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T21:50:58.990 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setuser ceph since I am not root 2024-04-26T21:50:58.990 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setgroup ceph since I am not root 2024-04-26T21:50:58.990 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:58.988+0000 7f857f1a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:50:59.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:59.236+0000 7f857f1a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.328+0000 7f52cf200700 1 Processor -- start 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.329+0000 7f52cf200700 1 -- start start 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.329+0000 7f52cf200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.329+0000 7f52cf200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810b800 con 0x7f52c810aea0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.330+0000 7f52c7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.330+0000 7f52c7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60274/0 (socket says 172.21.15.12:60274) 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.330+0000 7f52c7e00700 1 -- 172.21.15.12:0/2936780562 learned_addr learned my addr 172.21.15.12:0/2936780562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.330+0000 7f52c7e00700 1 -- 172.21.15.12:0/2936780562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c810c090 con 0x7f52c810aea0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.330+0000 7f52c7e00700 1 --2- 172.21.15.12:0/2936780562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9540c3cd4c6806 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.331+0000 7f52c6a00700 1 -- 172.21.15.12:0/2936780562 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8004160 con 0x7f52c810aea0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.331+0000 7f52c6a00700 1 -- 172.21.15.12:0/2936780562 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52b8004300 con 0x7f52c810aea0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.331+0000 7f52c6a00700 1 -- 172.21.15.12:0/2936780562 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8011af0 con 0x7f52c810aea0 2024-04-26T21:50:59.357 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.331+0000 7f52cf200700 1 -- 172.21.15.12:0/2936780562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 msgr2=0x7f52c810b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.331+0000 7f52cf200700 1 --2- 172.21.15.12:0/2936780562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.332+0000 7f52cf200700 1 -- 172.21.15.12:0/2936780562 shutdown_connections 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.332+0000 7f52cf200700 1 --2- 172.21.15.12:0/2936780562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c810b2c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.332+0000 7f52cf200700 1 -- 172.21.15.12:0/2936780562 >> 172.21.15.12:0/2936780562 conn(0x7f52c807e2e0 msgr2=0x7f52c807e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.332+0000 7f52cf200700 1 -- 172.21.15.12:0/2936780562 shutdown_connections 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.332+0000 7f52cf200700 1 -- 172.21.15.12:0/2936780562 wait complete. 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52cf200700 1 Processor -- start 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52cf200700 1 -- start start 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52cf200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52cf200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810f350 con 0x7f52c810aea0 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52c7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.333+0000 7f52c7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60284/0 (socket says 172.21.15.12:60284) 2024-04-26T21:50:59.358 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.334+0000 7f52c7e00700 1 -- 172.21.15.12:0/3171583166 learned_addr learned my addr 172.21.15.12:0/3171583166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.334+0000 7f52c7e00700 1 -- 172.21.15.12:0/3171583166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c819f410 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.334+0000 7f52c7e00700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f52b80117b0 tx=0x7f52b8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.334+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8003af0 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.335+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f52b8020070 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.335+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b8019630 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.335+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c819f6a0 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.335+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a2390 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.336+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f52b801a9c0 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.336+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8064a80 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.336+0000 7f52c4c00700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 0x7f52ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.336+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f52b804dae0 con 0x7f52c810aea0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.337+0000 7f52c7400700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 0x7f52ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.337+0000 7f52c7400700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 0x7f52ac03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:59.359 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.342+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f52b80042a0 con 0x7f52c810aea0 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.637+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f52c81a2560 con 0x7f52c810aea0 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.691+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f52c81a2560 con 0x7f52c810aea0 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.695+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 msgr2=0x7f52ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.695+0000 7f52cf200700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 0x7f52ac03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4006e40 comp rx=0 tx=0).stop 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.695+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 msgr2=0x7f52c819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.696+0000 7f52c4c00700 1 -- 172.21.15.12:0/3171583166 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f52b801a430 con 0x7f52c810aea0 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.697+0000 7f52cf200700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f52b80117b0 tx=0x7f52b8011b70 comp rx=0 tx=0).stop 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.697+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 shutdown_connections 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.697+0000 7f52cf200700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f52ac0395b0 0x7f52ac03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.697+0000 7f52cf200700 1 --2- 172.21.15.12:0/3171583166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52c810aea0 0x7f52c819eed0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.697+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 >> 172.21.15.12:0/3171583166 conn(0x7f52c807e2e0 msgr2=0x7f52c8108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.698+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 shutdown_connections 2024-04-26T21:50:59.360 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:58.698+0000 7f52cf200700 1 -- 172.21.15.12:0/3171583166 wait complete. 2024-04-26T21:50:59.641 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:59.372+0000 7f857f1a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:50:59.898 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3171583166' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T21:50:59.898 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-mon[28184]: mgrmap e5: x(active, since 5s) 2024-04-26T21:50:59.898 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:59.787+0000 7f857f1a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:51:00.177 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:50:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:50:59.907+0000 7f857f1a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:51:00.967 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:00 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1497467943' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:51:00.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:51:00.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T21:51:00.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:51:00.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:51:00.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:51:00.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.967+0000 7f22bac00700 1 Processor -- start 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.967+0000 7f22bac00700 1 -- start start 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.968+0000 7f22bac00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.968+0000 7f22bac00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22b407d8c0 con 0x7f22b407ee90 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.968+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.968+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38430/0 (socket says 172.21.15.12:38430) 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.968+0000 7f22b3e00700 1 -- 172.21.15.12:0/2061479319 learned_addr learned my addr 172.21.15.12:0/2061479319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.969+0000 7f22b3e00700 1 -- 172.21.15.12:0/2061479319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b407da40 con 0x7f22b407ee90 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.969+0000 7f22b3e00700 1 --2- 172.21.15.12:0/2061479319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22a40098a0 tx=0x7f22a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16850d3b983b9e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.969+0000 7f22b2a00700 1 -- 172.21.15.12:0/2061479319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a4004160 con 0x7f22b407ee90 2024-04-26T21:51:00.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.969+0000 7f22b2a00700 1 -- 172.21.15.12:0/2061479319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f22a4004300 con 0x7f22b407ee90 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.969+0000 7f22b2a00700 1 -- 172.21.15.12:0/2061479319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a4011af0 con 0x7f22b407ee90 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.970+0000 7f22bac00700 1 -- 172.21.15.12:0/2061479319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 msgr2=0x7f22b407d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.970+0000 7f22bac00700 1 --2- 172.21.15.12:0/2061479319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22a40098a0 tx=0x7f22a4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.970+0000 7f22bac00700 1 -- 172.21.15.12:0/2061479319 shutdown_connections 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.970+0000 7f22bac00700 1 --2- 172.21.15.12:0/2061479319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b407d2f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.970+0000 7f22bac00700 1 -- 172.21.15.12:0/2061479319 >> 172.21.15.12:0/2061479319 conn(0x7f22b41042a0 msgr2=0x7f22b41066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.971+0000 7f22bac00700 1 -- 172.21.15.12:0/2061479319 shutdown_connections 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.971+0000 7f22bac00700 1 -- 172.21.15.12:0/2061479319 wait complete. 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.972+0000 7f22bac00700 1 Processor -- start 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.972+0000 7f22bac00700 1 -- start start 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22bac00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22bac00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22b410f290 con 0x7f22b407ee90 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22b3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38442/0 (socket says 172.21.15.12:38442) 2024-04-26T21:51:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22b3e00700 1 -- 172.21.15.12:0/1497467943 learned_addr learned my addr 172.21.15.12:0/1497467943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.973+0000 7f22b3e00700 1 -- 172.21.15.12:0/1497467943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b419f320 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22b3e00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011ac0 tx=0x7f22a4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a4003b20 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f22a401a8c0 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22a40248d0 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b419f5b0 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.974+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41a22a0 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.975+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f22a4022020 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.975+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4064a80 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.976+0000 7f22b0c00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 0x7f229803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.976+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22a404d4d0 con 0x7f22b407ee90 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.976+0000 7f22b3400700 1 -- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 msgr2=0x7f229803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.976+0000 7f22b3400700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 0x7f229803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:51:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:50:59.981+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22a401ab20 con 0x7f22b407ee90 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.177+0000 7f22b3400700 1 -- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 msgr2=0x7f229803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.177+0000 7f22b3400700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 0x7f229803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.275+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f22b41a2470 con 0x7f22b407ee90 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.276+0000 7f22b0c00700 1 -- 172.21.15.12:0/1497467943 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f22b41a2470 con 0x7f22b407ee90 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 msgr2=0x7f229803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 0x7f229803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 msgr2=0x7f22b419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011ac0 tx=0x7f22a4011ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 shutdown_connections 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7f2298039600 0x7f229803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 --2- 172.21.15.12:0/1497467943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b407ee90 0x7f22b419ede0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.278+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 >> 172.21.15.12:0/1497467943 conn(0x7f22b41042a0 msgr2=0x7f22b4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.279+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 shutdown_connections 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:00.279+0000 7f22bac00700 1 -- 172.21.15.12:0/1497467943 wait complete. 2024-04-26T21:51:00.992 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-04-26T21:51:00.993 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-04-26T21:51:01.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:01.559+0000 7f857f1a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:51:01.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:01.791+0000 7f857f1a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:51:02.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:02.011+0000 7f857f1a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:51:02.787 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:02.402+0000 7f857f1a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:51:02.787 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:02.528+0000 7f857f1a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:51:03.050 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:02.786+0000 7f857f1a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:51:03.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:03.048+0000 7f857f1a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:51:04.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:03.861+0000 7f857f1a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:51:04.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:04 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:04.430+0000 7f857f1a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:51:04.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:04 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:04.564+0000 7f857f1a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:51:05.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:04 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:04.840+0000 7f857f1a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:51:05.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:05 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:05.505+0000 7f857f1a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:51:05.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:05 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:05.633+0000 7f857f1a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:51:06.277 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:05 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:05.963+0000 7f857f1a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:51:06.277 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:06 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:06.145+0000 7f857f1a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:51:06.277 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:06 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:06.276+0000 7f857f1a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:51:06.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:06 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:06.431+0000 7f857f1a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:51:07.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:06 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:06.805+0000 7f857f1a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:51:07.886 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:07.590+0000 7f857f1a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:51:08.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:07.885+0000 7f857f1a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:51:08.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: Active manager daemon x restarted 2024-04-26T21:51:08.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: Activating manager daemon x 2024-04-26T21:51:08.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: mgrmap e6: x(active, starting, since 0.00532122s) 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: Manager daemon x is now available 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:08.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:07 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: Found migration_current of "None". Setting to last migration. 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:09.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:09.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:08 smithi012 ceph-mon[28184]: mgrmap e7: x(active, since 1.01113s) 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.547+0000 7fabfe000700 1 Processor -- start 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.547+0000 7fabfe000700 1 -- start start 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.547+0000 7fabfe000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.547+0000 7fabfe000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fabf807d8a0 con 0x7fabf807ee70 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.548+0000 7fabf7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.548+0000 7fabf7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38454/0 (socket says 172.21.15.12:38454) 2024-04-26T21:51:09.671 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.548+0000 7fabf7400700 1 -- 172.21.15.12:0/3593818722 learned_addr learned my addr 172.21.15.12:0/3593818722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.548+0000 7fabf7400700 1 -- 172.21.15.12:0/3593818722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf807da20 con 0x7fabf807ee70 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.549+0000 7fabf7400700 1 --2- 172.21.15.12:0/3593818722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098a0 tx=0x7fabe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc73debd647cbd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.549+0000 7fabf6000700 1 -- 172.21.15.12:0/3593818722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe8004160 con 0x7fabf807ee70 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.549+0000 7fabf6000700 1 -- 172.21.15.12:0/3593818722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fabe8004300 con 0x7fabf807ee70 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.549+0000 7fabf6000700 1 -- 172.21.15.12:0/3593818722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe8011af0 con 0x7fabf807ee70 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 -- 172.21.15.12:0/3593818722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 msgr2=0x7fabf807d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 --2- 172.21.15.12:0/3593818722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098a0 tx=0x7fabe8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 -- 172.21.15.12:0/3593818722 shutdown_connections 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 --2- 172.21.15.12:0/3593818722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf807d2d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 -- 172.21.15.12:0/3593818722 >> 172.21.15.12:0/3593818722 conn(0x7fabf81042a0 msgr2=0x7fabf81066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 -- 172.21.15.12:0/3593818722 shutdown_connections 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.550+0000 7fabfe000700 1 -- 172.21.15.12:0/3593818722 wait complete. 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.551+0000 7fabfe000700 1 Processor -- start 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.551+0000 7fabfe000700 1 -- start start 2024-04-26T21:51:09.672 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabfe000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabfe000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fabf810f250 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabf7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabf7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38462/0 (socket says 172.21.15.12:38462) 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabf7400700 1 -- 172.21.15.12:0/2797118864 learned_addr learned my addr 172.21.15.12:0/2797118864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.552+0000 7fabf7400700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf819f320 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabf7400700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fabe8011ac0 tx=0x7fabe8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe8003b20 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fabe801a8c0 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe80248d0 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf819f5b0 con 0x7fabf807ee70 2024-04-26T21:51:09.673 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.553+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf81a22a0 con 0x7fabf807ee70 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.554+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fabe8022020 con 0x7fabf807ee70 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.555+0000 7fabefe00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.555+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabdc03c1d0 con 0x7fabdc039600 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.555+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabe804d4c0 con 0x7fabf807ee70 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.555+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.555+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.756+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:01.756+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:02.157+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:02.157+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:02.958+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.674 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:02.958+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:04.560+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:04.560+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:07.763+0000 7fabf6a00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1377681366 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:07.763+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:07.893+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fabe802d3f0 con 0x7fabf807ee70 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:07.894+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 msgr2=0x7fabdc03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:07.894+0000 7fabefe00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.899+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fabe804f260 con 0x7fabf807ee70 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.900+0000 7fabefe00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 0x7fabdc03f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.900+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabe801ac10 con 0x7fabdc03cef0 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.901+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 0x7fabdc03f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.903+0000 7fabf6a00700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 0x7fabdc03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fabe4003a10 tx=0x7fabe40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.905+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fabe801ac10 con 0x7fabdc03cef0 2024-04-26T21:51:09.675 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.921+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fabf8064a80 con 0x7fabdc03cef0 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.921+0000 7fabefe00700 1 -- 172.21.15.12:0/2797118864 <== mgr.14120 v2:172.21.15.12:6800/2513368485 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fabf8064a80 con 0x7fabdc03cef0 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 msgr2=0x7fabdc03f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 0x7fabdc03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fabe4003a10 tx=0x7fabe40092b0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 msgr2=0x7fabf819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fabe8011ac0 tx=0x7fabe8011ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 shutdown_connections 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fabdc03cef0 0x7fabdc03f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf807ee70 0x7fabf819ede0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 --2- 172.21.15.12:0/2797118864 >> [v2:172.21.15.12:6800/1377681366,v1:172.21.15.12:6801/1377681366] conn(0x7fabdc039600 0x7fabdc03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.922+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 >> 172.21.15.12:0/2797118864 conn(0x7fabf81042a0 msgr2=0x7fabf8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.923+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 shutdown_connections 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:08.923+0000 7fabfe000700 1 -- 172.21.15.12:0/2797118864 wait complete. 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-04-26T21:51:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-04-26T21:51:10.200 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:09 smithi012 ceph-mon[28184]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:51:10.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:09 smithi012 ceph-mon[28184]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:51:10.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:09 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:09] ENGINE Bus STARTING 2024-04-26T21:51:10.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:09 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:09] ENGINE Serving on http://172.21.15.12:8765 2024-04-26T21:51:10.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:09 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:11.254 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:09] ENGINE Serving on https://172.21.15.12:7150 2024-04-26T21:51:11.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:09] ENGINE Bus STARTED 2024-04-26T21:51:11.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: mgrmap e8: x(active, since 2s) 2024-04-26T21:51:11.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:11.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:11.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:10 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:11.270 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.279+0000 7fd538600700 1 Processor -- start 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.280+0000 7fd538600700 1 -- start start 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.280+0000 7fd538600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.280+0000 7fd538600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd53007ce60 con 0x7fd5301097b0 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.281+0000 7fd535c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.281+0000 7fd535c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35604/0 (socket says 172.21.15.12:35604) 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.281+0000 7fd535c00700 1 -- 172.21.15.12:0/1004855644 learned_addr learned my addr 172.21.15.12:0/1004855644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.281+0000 7fd535c00700 1 -- 172.21.15.12:0/1004855644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53010c0e0 con 0x7fd5301097b0 2024-04-26T21:51:11.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.282+0000 7fd535c00700 1 --2- 172.21.15.12:0/1004855644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5240098a0 tx=0x7fd524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb65a097cc931f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.282+0000 7fd52fe00700 1 -- 172.21.15.12:0/1004855644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd524004160 con 0x7fd5301097b0 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.282+0000 7fd52fe00700 1 -- 172.21.15.12:0/1004855644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd524004300 con 0x7fd5301097b0 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.282+0000 7fd52fe00700 1 -- 172.21.15.12:0/1004855644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd524011af0 con 0x7fd5301097b0 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 -- 172.21.15.12:0/1004855644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 msgr2=0x7fd53010bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 --2- 172.21.15.12:0/1004855644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5240098a0 tx=0x7fd524009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 -- 172.21.15.12:0/1004855644 shutdown_connections 2024-04-26T21:51:11.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 --2- 172.21.15.12:0/1004855644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53010bba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 -- 172.21.15.12:0/1004855644 >> 172.21.15.12:0/1004855644 conn(0x7fd5301031a0 msgr2=0x7fd5301055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 -- 172.21.15.12:0/1004855644 shutdown_connections 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.283+0000 7fd538600700 1 -- 172.21.15.12:0/1004855644 wait complete. 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.284+0000 7fd538600700 1 Processor -- start 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.284+0000 7fd538600700 1 -- start start 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd538600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd538600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd53010d050 con 0x7fd5301097b0 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd535c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd535c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35612/0 (socket says 172.21.15.12:35612) 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd535c00700 1 -- 172.21.15.12:0/2813284712 learned_addr learned my addr 172.21.15.12:0/2813284712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.285+0000 7fd535c00700 1 -- 172.21.15.12:0/2813284712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53019f360 con 0x7fd5301097b0 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd535c00700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd524011ac0 tx=0x7fd524011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd524003b20 con 0x7fd5301097b0 2024-04-26T21:51:11.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd52401a8c0 con 0x7fd5301097b0 2024-04-26T21:51:11.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53019f5f0 con 0x7fd5301097b0 2024-04-26T21:51:11.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5240248d0 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.286+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5301a22e0 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.287+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd524022020 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.287+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd530064a80 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.288+0000 7fd52e000700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 0x7fd51803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.288+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd52404d8f0 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.288+0000 7fd535200700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 0x7fd51803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.289+0000 7fd535200700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 0x7fd51803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006fd0 tx=0x7fd51c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.294+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd52401ab20 con 0x7fd5301097b0 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.543+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd5301043a0 con 0x7fd518039660 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.558+0000 7fd52e000700 1 -- 172.21.15.12:0/2813284712 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5301043a0 con 0x7fd518039660 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 msgr2=0x7fd51803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 0x7fd51803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006fd0 tx=0x7fd51c006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 msgr2=0x7fd53019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd524011ac0 tx=0x7fd524011ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 shutdown_connections 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fd518039660 0x7fd51803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 --2- 172.21.15.12:0/2813284712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5301097b0 0x7fd53019ee20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 >> 172.21.15.12:0/2813284712 conn(0x7fd5301031a0 msgr2=0x7fd530103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.561+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 shutdown_connections 2024-04-26T21:51:11.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:10.562+0000 7fd538600700 1 -- 172.21.15.12:0/2813284712 wait complete. 2024-04-26T21:51:12.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.919+0000 7f05afa00700 1 Processor -- start 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.919+0000 7f05afa00700 1 -- start start 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.920+0000 7f05afa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.920+0000 7f05afa00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81096e0 con 0x7f05a8108cf0 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.920+0000 7f05ad000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.920+0000 7f05ad000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35618/0 (socket says 172.21.15.12:35618) 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.920+0000 7f05ad000700 1 -- 172.21.15.12:0/1090213535 learned_addr learned my addr 172.21.15.12:0/1090213535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:12.817 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.921+0000 7f05ad000700 1 -- 172.21.15.12:0/1090213535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a8109ef0 con 0x7f05a8108cf0 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.921+0000 7f05ad000700 1 --2- 172.21.15.12:0/1090213535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f059c0098a0 tx=0x7f059c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b26a70f80ab9342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.921+0000 7f05a7400700 1 -- 172.21.15.12:0/1090213535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f059c004160 con 0x7f05a8108cf0 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.921+0000 7f05a7400700 1 -- 172.21.15.12:0/1090213535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f059c004300 con 0x7f05a8108cf0 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.921+0000 7f05a7400700 1 -- 172.21.15.12:0/1090213535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f059c011af0 con 0x7f05a8108cf0 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.922+0000 7f05afa00700 1 -- 172.21.15.12:0/1090213535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 msgr2=0x7f05a8109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.922+0000 7f05afa00700 1 --2- 172.21.15.12:0/1090213535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f059c0098a0 tx=0x7f059c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.923+0000 7f05afa00700 1 -- 172.21.15.12:0/1090213535 shutdown_connections 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.923+0000 7f05afa00700 1 --2- 172.21.15.12:0/1090213535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a8109110 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.923+0000 7f05afa00700 1 -- 172.21.15.12:0/1090213535 >> 172.21.15.12:0/1090213535 conn(0x7f05a8104290 msgr2=0x7f05a81066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.923+0000 7f05afa00700 1 -- 172.21.15.12:0/1090213535 shutdown_connections 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.923+0000 7f05afa00700 1 -- 172.21.15.12:0/1090213535 wait complete. 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.924+0000 7f05afa00700 1 Processor -- start 2024-04-26T21:51:12.818 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.924+0000 7f05afa00700 1 -- start start 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.924+0000 7f05afa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.924+0000 7f05afa00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05a810d2a0 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.925+0000 7f05ad000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.925+0000 7f05ad000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35632/0 (socket says 172.21.15.12:35632) 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.925+0000 7f05ad000700 1 -- 172.21.15.12:0/937162134 learned_addr learned my addr 172.21.15.12:0/937162134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.925+0000 7f05ad000700 1 -- 172.21.15.12:0/937162134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a819f3e0 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.925+0000 7f05ad000700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f059c009fb0 tx=0x7f059c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.926+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f059c0036a0 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.926+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f059c01a430 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.926+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f059c019550 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.926+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a819f670 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.926+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81a2360 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.927+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f059c01a8e0 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.927+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a8064a80 con 0x7f05a8108cf0 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.927+0000 7f05a5600700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 0x7f059003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.819 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.928+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f059c04d9e0 con 0x7f05a8108cf0 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.928+0000 7f05a7e00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 0x7f059003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.928+0000 7f05a7e00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 0x7f059003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:11.934+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f059c01a690 con 0x7f05a8108cf0 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.164+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f05a81054a0 con 0x7f0590039560 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.165+0000 7f05a5600700 1 -- 172.21.15.12:0/937162134 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f05a81054a0 con 0x7f0590039560 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 msgr2=0x7f059003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 0x7f059003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 msgr2=0x7f05a819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f059c009fb0 tx=0x7f059c011940 comp rx=0 tx=0).stop 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 shutdown_connections 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f0590039560 0x7f059003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 --2- 172.21.15.12:0/937162134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05a8108cf0 0x7f05a819eea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.820 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 >> 172.21.15.12:0/937162134 conn(0x7f05a8104290 msgr2=0x7f05a8104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 shutdown_connections 2024-04-26T21:51:12.821 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:12.168+0000 7f05afa00700 1 -- 172.21.15.12:0/937162134 wait complete. 2024-04-26T21:51:12.821 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-04-26T21:51:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:12 smithi012 ceph-mon[28184]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-mon[28184]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-mon[28184]: Generating ssh key... 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: Generating public/private rsa key pair. 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: Your identification has been saved in /tmp/tmpjhd3zrz5/key. 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: Your public key has been saved in /tmp/tmpjhd3zrz5/key.pub. 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: The key fingerprint is: 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: SHA256:d/jkfU8bPEqtKc7+EplMHhjxxw8rUgkdo23HdhAjZ3U ceph-d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: The key's randomart image is: 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: +---[RSA 3072]----+ 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | oo+.*o.E| 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | .=.O o .| 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | .o* B . | 2024-04-26T21:51:14.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | .o++ = | 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | S.*.*. . | 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | ..@..o | 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | +..=o| 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | .o. +o=| 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: | o++= ..| 2024-04-26T21:51:14.075 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: +----[SHA256]-----+ 2024-04-26T21:51:14.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.435+0000 7fe7f6c00700 1 Processor -- start 2024-04-26T21:51:14.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.435+0000 7fe7f6c00700 1 -- start start 2024-04-26T21:51:14.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.436+0000 7fe7f6c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.653 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.436+0000 7fe7f6c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f007d8c0 con 0x7fe7f007ee90 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.436+0000 7fe7efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.436+0000 7fe7efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35642/0 (socket says 172.21.15.12:35642) 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.437+0000 7fe7efe00700 1 -- 172.21.15.12:0/4026570112 learned_addr learned my addr 172.21.15.12:0/4026570112 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.437+0000 7fe7efe00700 1 -- 172.21.15.12:0/4026570112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f007da40 con 0x7fe7f007ee90 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.437+0000 7fe7efe00700 1 --2- 172.21.15.12:0/4026570112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00098a0 tx=0x7fe7e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212aebaf0d0352e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:14.654 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.438+0000 7fe7eea00700 1 -- 172.21.15.12:0/4026570112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e0004160 con 0x7fe7f007ee90 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.438+0000 7fe7eea00700 1 -- 172.21.15.12:0/4026570112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7e0004300 con 0x7fe7f007ee90 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.438+0000 7fe7eea00700 1 -- 172.21.15.12:0/4026570112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e0011af0 con 0x7fe7f007ee90 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.438+0000 7fe7f6c00700 1 -- 172.21.15.12:0/4026570112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 msgr2=0x7fe7f007d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.438+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/4026570112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00098a0 tx=0x7fe7e0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.439+0000 7fe7f6c00700 1 -- 172.21.15.12:0/4026570112 shutdown_connections 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.439+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/4026570112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f007d2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.439+0000 7fe7f6c00700 1 -- 172.21.15.12:0/4026570112 >> 172.21.15.12:0/4026570112 conn(0x7fe7f01042a0 msgr2=0x7fe7f01066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.439+0000 7fe7f6c00700 1 -- 172.21.15.12:0/4026570112 shutdown_connections 2024-04-26T21:51:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.439+0000 7fe7f6c00700 1 -- 172.21.15.12:0/4026570112 wait complete. 2024-04-26T21:51:14.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.440+0000 7fe7f6c00700 1 Processor -- start 2024-04-26T21:51:14.657 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.440+0000 7fe7f6c00700 1 -- start start 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.440+0000 7fe7f6c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.440+0000 7fe7f6c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f010f290 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.441+0000 7fe7efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.441+0000 7fe7efe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35656/0 (socket says 172.21.15.12:35656) 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.441+0000 7fe7efe00700 1 -- 172.21.15.12:0/1040608726 learned_addr learned my addr 172.21.15.12:0/1040608726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.441+0000 7fe7efe00700 1 -- 172.21.15.12:0/1040608726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f019f320 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.441+0000 7fe7efe00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0011ac0 tx=0x7fe7e0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.442+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e0003b20 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.442+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f019f5b0 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.442+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7e001a8c0 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.442+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e00248d0 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.442+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01a22a0 con 0x7fe7f007ee90 2024-04-26T21:51:14.659 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.443+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe7e0022020 con 0x7fe7f007ee90 2024-04-26T21:51:14.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.443+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0064a80 con 0x7fe7f007ee90 2024-04-26T21:51:14.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.443+0000 7fe7ecc00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 0x7fe7d403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.660 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.443+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7e004d880 con 0x7fe7f007ee90 2024-04-26T21:51:14.661 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.444+0000 7fe7ef400700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 0x7fe7d403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.444+0000 7fe7ef400700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 0x7fe7d403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.449+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7e000f2e0 con 0x7fe7f007ee90 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.684+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe7f01054e0 con 0x7fe7d40395b0 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.955+0000 7fe7ecc00700 1 -- 172.21.15.12:0/1040608726 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7f01054e0 con 0x7fe7d40395b0 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 msgr2=0x7fe7d403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 0x7fe7d403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 msgr2=0x7fe7f019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0011ac0 tx=0x7fe7e0011ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 shutdown_connections 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.958+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe7d40395b0 0x7fe7d403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.959+0000 7fe7f6c00700 1 --2- 172.21.15.12:0/1040608726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7f007ee90 0x7fe7f019ede0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.959+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 >> 172.21.15.12:0/1040608726 conn(0x7fe7f01042a0 msgr2=0x7fe7f0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:14.662 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.959+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 shutdown_connections 2024-04-26T21:51:14.663 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:13.959+0000 7fe7f6c00700 1 -- 172.21.15.12:0/1040608726 wait complete. 2024-04-26T21:51:15.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:14 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:15.220 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:14 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:16.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:15 smithi012 ceph-mon[28184]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.288+0000 7feabba00700 1 Processor -- start 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.289+0000 7feabba00700 1 -- start start 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.289+0000 7feabba00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.289+0000 7feabba00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feab410b800 con 0x7feab410aea0 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.289+0000 7feab9000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.290+0000 7feab9000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35662/0 (socket says 172.21.15.12:35662) 2024-04-26T21:51:16.254 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.290+0000 7feab9000700 1 -- 172.21.15.12:0/2020590050 learned_addr learned my addr 172.21.15.12:0/2020590050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.290+0000 7feab9000700 1 -- 172.21.15.12:0/2020590050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab410c090 con 0x7feab410aea0 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.290+0000 7feab9000700 1 --2- 172.21.15.12:0/2020590050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098a0 tx=0x7feaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47fc0656c8fdc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feab3e00700 1 -- 172.21.15.12:0/2020590050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4004160 con 0x7feab410aea0 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feab3e00700 1 -- 172.21.15.12:0/2020590050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feaa4004300 con 0x7feab410aea0 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feab3e00700 1 -- 172.21.15.12:0/2020590050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4011af0 con 0x7feab410aea0 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feabba00700 1 -- 172.21.15.12:0/2020590050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 msgr2=0x7feab410b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feabba00700 1 --2- 172.21.15.12:0/2020590050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098a0 tx=0x7feaa4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feabba00700 1 -- 172.21.15.12:0/2020590050 shutdown_connections 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feabba00700 1 --2- 172.21.15.12:0/2020590050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab410b2c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.291+0000 7feabba00700 1 -- 172.21.15.12:0/2020590050 >> 172.21.15.12:0/2020590050 conn(0x7feab407e2e0 msgr2=0x7feab407e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.292+0000 7feabba00700 1 -- 172.21.15.12:0/2020590050 shutdown_connections 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.292+0000 7feabba00700 1 -- 172.21.15.12:0/2020590050 wait complete. 2024-04-26T21:51:16.255 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feabba00700 1 Processor -- start 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feabba00700 1 -- start start 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feabba00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feabba00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feab410f350 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feab9000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feab9000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35668/0 (socket says 172.21.15.12:35668) 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.293+0000 7feab9000700 1 -- 172.21.15.12:0/902476721 learned_addr learned my addr 172.21.15.12:0/902476721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.294+0000 7feab9000700 1 -- 172.21.15.12:0/902476721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab419f410 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.294+0000 7feab9000700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feaa40117b0 tx=0x7feaa4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.294+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4003af0 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.294+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feaa4020070 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.295+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4019630 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.295+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab419f6a0 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.295+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a2390 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.296+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7feaa401a9c0 con 0x7feab410aea0 2024-04-26T21:51:16.256 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.296+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab4064a80 con 0x7feab410aea0 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.297+0000 7feab2000700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 0x7fea9803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.297+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feaa404dbf0 con 0x7feab410aea0 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.297+0000 7feaabe00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 0x7fea9803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.298+0000 7feaabe00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 0x7fea9803bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fea9c006fd0 tx=0x7fea9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.302+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feaa400e260 con 0x7feab410aea0 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.531+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7feab4108c40 con 0x7fea98039660 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.533+0000 7feab2000700 1 -- 172.21.15.12:0/902476721 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7feab4108c40 con 0x7fea98039660 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 msgr2=0x7fea9803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 0x7fea9803bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fea9c006fd0 tx=0x7fea9c006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 msgr2=0x7feab419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feaa40117b0 tx=0x7feaa4011b70 comp rx=0 tx=0).stop 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 shutdown_connections 2024-04-26T21:51:16.257 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fea98039660 0x7fea9803bb20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 --2- 172.21.15.12:0/902476721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab410aea0 0x7feab419eed0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.536+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 >> 172.21.15.12:0/902476721 conn(0x7feab407e2e0 msgr2=0x7feab4108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.537+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 shutdown_connections 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:15.537+0000 7feabba00700 1 -- 172.21.15.12:0/902476721 wait complete. 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:51:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T21:51:16.260 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-04-26T21:51:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:17 smithi012 ceph-mon[28184]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:19.027 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:18 smithi012 ceph-mon[28184]: Deploying cephadm binary to smithi012 2024-04-26T21:51:24.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-04-26T21:51:24.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.847+0000 7fe833800700 1 Processor -- start 2024-04-26T21:51:24.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.847+0000 7fe833800700 1 -- start start 2024-04-26T21:51:24.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.848+0000 7fe833800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:24.236 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.848+0000 7fe833800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c10b800 con 0x7fe82c10aea0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.848+0000 7fe830e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.848+0000 7fe830e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35672/0 (socket says 172.21.15.12:35672) 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.848+0000 7fe830e00700 1 -- 172.21.15.12:0/1791365132 learned_addr learned my addr 172.21.15.12:0/1791365132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.849+0000 7fe830e00700 1 -- 172.21.15.12:0/1791365132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c10c090 con 0x7fe82c10aea0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.849+0000 7fe830e00700 1 --2- 172.21.15.12:0/1791365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098a0 tx=0x7fe820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607294c68366ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.849+0000 7fe82b400700 1 -- 172.21.15.12:0/1791365132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820004160 con 0x7fe82c10aea0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe82b400700 1 -- 172.21.15.12:0/1791365132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe820004300 con 0x7fe82c10aea0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe82b400700 1 -- 172.21.15.12:0/1791365132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820011af0 con 0x7fe82c10aea0 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe833800700 1 -- 172.21.15.12:0/1791365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 msgr2=0x7fe82c10b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe833800700 1 --2- 172.21.15.12:0/1791365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098a0 tx=0x7fe820009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:24.237 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe833800700 1 -- 172.21.15.12:0/1791365132 shutdown_connections 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe833800700 1 --2- 172.21.15.12:0/1791365132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c10b2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.850+0000 7fe833800700 1 -- 172.21.15.12:0/1791365132 >> 172.21.15.12:0/1791365132 conn(0x7fe82c07e2e0 msgr2=0x7fe82c07e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.851+0000 7fe833800700 1 -- 172.21.15.12:0/1791365132 shutdown_connections 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.851+0000 7fe833800700 1 -- 172.21.15.12:0/1791365132 wait complete. 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.852+0000 7fe833800700 1 Processor -- start 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.852+0000 7fe833800700 1 -- start start 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.852+0000 7fe833800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.852+0000 7fe833800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c10f350 con 0x7fe82c10aea0 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.853+0000 7fe830e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.853+0000 7fe830e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35678/0 (socket says 172.21.15.12:35678) 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.853+0000 7fe830e00700 1 -- 172.21.15.12:0/1325988483 learned_addr learned my addr 172.21.15.12:0/1325988483 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.853+0000 7fe830e00700 1 -- 172.21.15.12:0/1325988483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c19f410 con 0x7fe82c10aea0 2024-04-26T21:51:24.238 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.853+0000 7fe830e00700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8200117b0 tx=0x7fe820011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.854+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820003af0 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.854+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe820020070 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.854+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820019630 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.854+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c19f6a0 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.854+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c1a2390 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.855+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe82001a9c0 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.855+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82c064a80 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.856+0000 7fe829600700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 0x7fe81403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.856+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe82004dbf0 con 0x7fe82c10aea0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.856+0000 7fe82be00700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 0x7fe81403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.857+0000 7fe82be00700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 0x7fe81403bb20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe818006fd0 tx=0x7fe818006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:16.861+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8200042a0 con 0x7fe82c10aea0 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:17.097+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) -- 0x7fe82c108c40 con 0x7fe814039660 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.465+0000 7fe829600700 1 -- 172.21.15.12:0/1325988483 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe82c108c40 con 0x7fe814039660 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.469+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 msgr2=0x7fe81403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.469+0000 7fe833800700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 0x7fe81403bb20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe818006fd0 tx=0x7fe818006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.469+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 msgr2=0x7fe82c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.469+0000 7fe833800700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8200117b0 tx=0x7fe820011b70 comp rx=0 tx=0).stop 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 shutdown_connections 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7fe814039660 0x7fe81403bb20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 --2- 172.21.15.12:0/1325988483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c10aea0 0x7fe82c19eed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 >> 172.21.15.12:0/1325988483 conn(0x7fe82c07e2e0 msgr2=0x7fe82c108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 shutdown_connections 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:23.470+0000 7fe833800700 1 -- 172.21.15.12:0/1325988483 wait complete. 2024-04-26T21:51:24.240 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-04-26T21:51:24.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:24 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:24.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:24 smithi012 ceph-mon[28184]: Added host smithi012 2024-04-26T21:51:24.503 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:24 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:26.391 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:26 smithi012 ceph-mon[28184]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:26.391 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:26 smithi012 ceph-mon[28184]: Saving service mon spec with placement count:5 2024-04-26T21:51:26.391 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:26 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:26.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T21:51:26.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.833+0000 7f0812400700 1 Processor -- start 2024-04-26T21:51:26.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.833+0000 7f0812400700 1 -- start start 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.834+0000 7f0812400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.834+0000 7f0812400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f080c083560 con 0x7f080c10fce0 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.834+0000 7f080b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.834+0000 7f080b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55924/0 (socket says 172.21.15.12:55924) 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.834+0000 7f080b400700 1 -- 172.21.15.12:0/548184398 learned_addr learned my addr 172.21.15.12:0/548184398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:26.407 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.835+0000 7f080b400700 1 -- 172.21.15.12:0/548184398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c112610 con 0x7f080c10fce0 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.835+0000 7f080b400700 1 --2- 172.21.15.12:0/548184398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098a0 tx=0x7f07fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556f4639231ad5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.836+0000 7f080a000700 1 -- 172.21.15.12:0/548184398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc004160 con 0x7f080c10fce0 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.836+0000 7f080a000700 1 -- 172.21.15.12:0/548184398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f07fc004300 con 0x7f080c10fce0 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.836+0000 7f080a000700 1 -- 172.21.15.12:0/548184398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc011af0 con 0x7f080c10fce0 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.836+0000 7f0812400700 1 -- 172.21.15.12:0/548184398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 msgr2=0x7f080c1120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.836+0000 7f0812400700 1 --2- 172.21.15.12:0/548184398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098a0 tx=0x7f07fc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.837+0000 7f0812400700 1 -- 172.21.15.12:0/548184398 shutdown_connections 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.837+0000 7f0812400700 1 --2- 172.21.15.12:0/548184398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1120d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.837+0000 7f0812400700 1 -- 172.21.15.12:0/548184398 >> 172.21.15.12:0/548184398 conn(0x7f080c07e2e0 msgr2=0x7f080c080700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.837+0000 7f0812400700 1 -- 172.21.15.12:0/548184398 shutdown_connections 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.837+0000 7f0812400700 1 -- 172.21.15.12:0/548184398 wait complete. 2024-04-26T21:51:26.408 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.838+0000 7f0812400700 1 Processor -- start 2024-04-26T21:51:26.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.838+0000 7f0812400700 1 -- start start 2024-04-26T21:51:26.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.838+0000 7f0812400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:26.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.838+0000 7f0812400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f080c113690 con 0x7f080c10fce0 2024-04-26T21:51:26.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.839+0000 7f080b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:26.409 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.839+0000 7f080b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55926/0 (socket says 172.21.15.12:55926) 2024-04-26T21:51:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.839+0000 7f080b400700 1 -- 172.21.15.12:0/2495096153 learned_addr learned my addr 172.21.15.12:0/2495096153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.839+0000 7f080b400700 1 -- 172.21.15.12:0/2495096153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c1a37d0 con 0x7f080c10fce0 2024-04-26T21:51:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.839+0000 7f080b400700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011ac0 tx=0x7f07fc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.840+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc003970 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.840+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f07fc003b10 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.840+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc024610 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.840+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1a3a60 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.840+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1a6750 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.841+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c064a80 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.841+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f07fc01a9e0 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.842+0000 7f0803e00700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 0x7f07f003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.842+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07fc04db10 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.842+0000 7f080aa00700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 0x7f07f003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.843+0000 7f080aa00700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 0x7f07f003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f07f8006fd0 tx=0x7f07f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:24.848+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07fc04d460 con 0x7f080c10fce0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.104+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f080c07f530 con 0x7f07f00395b0 2024-04-26T21:51:26.413 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.113+0000 7f0803e00700 1 -- 172.21.15.12:0/2495096153 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f080c07f530 con 0x7f07f00395b0 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.115+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 msgr2=0x7f07f003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.115+0000 7f0812400700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 0x7f07f003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f07f8006fd0 tx=0x7f07f8006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.115+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 msgr2=0x7f080c1a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.115+0000 7f0812400700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011ac0 tx=0x7f07fc011ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 shutdown_connections 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f07f00395b0 0x7f07f003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 --2- 172.21.15.12:0/2495096153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f080c10fce0 0x7f080c1a3290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 >> 172.21.15.12:0/2495096153 conn(0x7f080c07e2e0 msgr2=0x7f080c07ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 shutdown_connections 2024-04-26T21:51:26.414 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:25.116+0000 7f0812400700 1 -- 172.21.15.12:0/2495096153 wait complete. 2024-04-26T21:51:26.415 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-04-26T21:51:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:27 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:27 smithi012 ceph-mon[28184]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:27 smithi012 ceph-mon[28184]: Saving service mgr spec with placement count:2 2024-04-26T21:51:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:27 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:27 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:28.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T21:51:28.427 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.503+0000 7f7b68000700 1 Processor -- start 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.503+0000 7f7b68000700 1 -- start start 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.504+0000 7f7b68000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.504+0000 7f7b68000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6007d900 con 0x7f7b6007eed0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.504+0000 7f7b65600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.504+0000 7f7b65600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55940/0 (socket says 172.21.15.12:55940) 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.504+0000 7f7b65600700 1 -- 172.21.15.12:0/1811267397 learned_addr learned my addr 172.21.15.12:0/1811267397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.505+0000 7f7b65600700 1 -- 172.21.15.12:0/1811267397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6007da80 con 0x7f7b6007eed0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.505+0000 7f7b65600700 1 --2- 172.21.15.12:0/1811267397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b540098a0 tx=0x7f7b54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b66b4e97a5dcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.506+0000 7f7b5fe00700 1 -- 172.21.15.12:0/1811267397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b54004160 con 0x7f7b6007eed0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.506+0000 7f7b5fe00700 1 -- 172.21.15.12:0/1811267397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b54004300 con 0x7f7b6007eed0 2024-04-26T21:51:28.428 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.506+0000 7f7b5fe00700 1 -- 172.21.15.12:0/1811267397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b54011a20 con 0x7f7b6007eed0 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.506+0000 7f7b68000700 1 -- 172.21.15.12:0/1811267397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 msgr2=0x7f7b6007d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.506+0000 7f7b68000700 1 --2- 172.21.15.12:0/1811267397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b540098a0 tx=0x7f7b54009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.507+0000 7f7b68000700 1 -- 172.21.15.12:0/1811267397 shutdown_connections 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.507+0000 7f7b68000700 1 --2- 172.21.15.12:0/1811267397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6007d330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.507+0000 7f7b68000700 1 -- 172.21.15.12:0/1811267397 >> 172.21.15.12:0/1811267397 conn(0x7f7b601042a0 msgr2=0x7f7b601066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.507+0000 7f7b68000700 1 -- 172.21.15.12:0/1811267397 shutdown_connections 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.507+0000 7f7b68000700 1 -- 172.21.15.12:0/1811267397 wait complete. 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.508+0000 7f7b68000700 1 Processor -- start 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.509+0000 7f7b68000700 1 -- start start 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.509+0000 7f7b68000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.509+0000 7f7b68000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010f2a0 con 0x7f7b6007eed0 2024-04-26T21:51:28.429 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.509+0000 7f7b65600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.510+0000 7f7b65600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55942/0 (socket says 172.21.15.12:55942) 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.510+0000 7f7b65600700 1 -- 172.21.15.12:0/3297441693 learned_addr learned my addr 172.21.15.12:0/3297441693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.510+0000 7f7b65600700 1 -- 172.21.15.12:0/3297441693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6019f2f0 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.510+0000 7f7b65600700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009520 tx=0x7f7b54011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.511+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b540039e0 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.511+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5401b8f0 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.511+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b54019770 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.511+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6019f580 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.511+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b601a2230 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.512+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7b5401b440 con 0x7f7b6007eed0 2024-04-26T21:51:28.430 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.512+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b60064a80 con 0x7f7b6007eed0 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.513+0000 7f7b5e000700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 0x7f7b4803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.513+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b5404df50 con 0x7f7b6007eed0 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.514+0000 7f7b64c00700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 0x7f7b4803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.514+0000 7f7b64c00700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 0x7f7b4803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.519+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7b5401f030 con 0x7f7b6007eed0 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.768+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7b60105430 con 0x7f7b48039600 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.775+0000 7f7b5e000700 1 -- 172.21.15.12:0/3297441693 <== mgr.14120 v2:172.21.15.12:6800/2513368485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7b60105430 con 0x7f7b48039600 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 msgr2=0x7f7b4803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 0x7f7b4803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 msgr2=0x7f7b6019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009520 tx=0x7f7b54011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.431 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 shutdown_connections 2024-04-26T21:51:28.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7b48039600 0x7f7b4803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 --2- 172.21.15.12:0/3297441693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b6007eed0 0x7f7b6019edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 >> 172.21.15.12:0/3297441693 conn(0x7f7b601042a0 msgr2=0x7f7b60104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:28.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 shutdown_connections 2024-04-26T21:51:28.432 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:27.778+0000 7f7b68000700 1 -- 172.21.15.12:0/3297441693 wait complete. 2024-04-26T21:51:29.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:28 smithi012 ceph-mon[28184]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:51:30.480 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.058+0000 7f5d53c00700 1 Processor -- start 2024-04-26T21:51:30.480 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.058+0000 7f5d53c00700 1 -- start start 2024-04-26T21:51:30.480 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.059+0000 7f5d53c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.059+0000 7f5d53c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c07d8a0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.059+0000 7f5d51200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.059+0000 7f5d51200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39702/0 (socket says 172.21.15.12:39702) 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.059+0000 7f5d51200700 1 -- 172.21.15.12:0/2625572441 learned_addr learned my addr 172.21.15.12:0/2625572441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.060+0000 7f5d51200700 1 -- 172.21.15.12:0/2625572441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c07da20 con 0x7f5d4c07ee70 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.060+0000 7f5d51200700 1 --2- 172.21.15.12:0/2625572441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098a0 tx=0x7f5d40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885576eb2490f851 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.060+0000 7f5d4b400700 1 -- 172.21.15.12:0/2625572441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d40004160 con 0x7f5d4c07ee70 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.060+0000 7f5d4b400700 1 -- 172.21.15.12:0/2625572441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d40004300 con 0x7f5d4c07ee70 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.060+0000 7f5d4b400700 1 -- 172.21.15.12:0/2625572441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d40011a20 con 0x7f5d4c07ee70 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.061+0000 7f5d53c00700 1 -- 172.21.15.12:0/2625572441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 msgr2=0x7f5d4c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.061+0000 7f5d53c00700 1 --2- 172.21.15.12:0/2625572441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098a0 tx=0x7f5d40009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.061+0000 7f5d53c00700 1 -- 172.21.15.12:0/2625572441 shutdown_connections 2024-04-26T21:51:30.481 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.061+0000 7f5d53c00700 1 --2- 172.21.15.12:0/2625572441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c07d2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.061+0000 7f5d53c00700 1 -- 172.21.15.12:0/2625572441 >> 172.21.15.12:0/2625572441 conn(0x7f5d4c1042a0 msgr2=0x7f5d4c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.062+0000 7f5d53c00700 1 -- 172.21.15.12:0/2625572441 shutdown_connections 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.062+0000 7f5d53c00700 1 -- 172.21.15.12:0/2625572441 wait complete. 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.063+0000 7f5d53c00700 1 Processor -- start 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.063+0000 7f5d53c00700 1 -- start start 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.063+0000 7f5d53c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.063+0000 7f5d53c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10f2a0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.063+0000 7f5d51200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.064+0000 7f5d51200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39704/0 (socket says 172.21.15.12:39704) 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.064+0000 7f5d51200700 1 -- 172.21.15.12:0/1968268245 learned_addr learned my addr 172.21.15.12:0/1968268245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.064+0000 7f5d51200700 1 -- 172.21.15.12:0/1968268245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c19f2f0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.482 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.064+0000 7f5d51200700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d40000c00 tx=0x7f5d40011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.064+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d40003aa0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.065+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4001b8f0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.065+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c19f580 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.065+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d40023870 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.065+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a2230 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.066+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c064a80 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.067+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5d40023b70 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.067+0000 7f5d49600700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 0x7f5d3403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.068+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5d4004dec0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.068+0000 7f5d4be00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 0x7f5d3403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.068+0000 7f5d4be00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 0x7f5d3403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.076+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5d4001f030 con 0x7f5d4c07ee70 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.320+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f5d4c1a2400 con 0x7f5d4c07ee70 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.326+0000 7f5d49600700 1 -- 172.21.15.12:0/1968268245 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f5d4002b3a0 con 0x7f5d4c07ee70 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 msgr2=0x7f5d3403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 0x7f5d3403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 msgr2=0x7f5d4c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d40000c00 tx=0x7f5d40011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 shutdown_connections 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f5d340395b0 0x7f5d3403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 --2- 172.21.15.12:0/1968268245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4c07ee70 0x7f5d4c19edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.330+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 >> 172.21.15.12:0/1968268245 conn(0x7f5d4c1042a0 msgr2=0x7f5d4c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.331+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 shutdown_connections 2024-04-26T21:51:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:29.331+0000 7f5d53c00700 1 -- 172.21.15.12:0/1968268245 wait complete. 2024-04-26T21:51:30.649 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:30 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1968268245' entity='client.admin' 2024-04-26T21:51:31.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:31 smithi012 ceph-mon[28184]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:51:31.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:31 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:31.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:31 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3360147295' entity='client.admin' 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.187+0000 7f8bd9e00700 1 Processor -- start 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.187+0000 7f8bd9e00700 1 -- start start 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.188+0000 7f8bd9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.188+0000 7f8bd9e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd407cec0 con 0x7f8bd4109750 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.188+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.189+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39712/0 (socket says 172.21.15.12:39712) 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.189+0000 7f8bd2a00700 1 -- 172.21.15.12:0/1666671340 learned_addr learned my addr 172.21.15.12:0/1666671340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.189+0000 7f8bd2a00700 1 -- 172.21.15.12:0/1666671340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd410c080 con 0x7f8bd4109750 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.189+0000 7f8bd2a00700 1 --2- 172.21.15.12:0/1666671340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40098a0 tx=0x7f8bc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265f2c6bcb3e843b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd2000700 1 -- 172.21.15.12:0/1666671340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc4004160 con 0x7f8bd4109750 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd2000700 1 -- 172.21.15.12:0/1666671340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc4004300 con 0x7f8bd4109750 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd2000700 1 -- 172.21.15.12:0/1666671340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc4011a20 con 0x7f8bd4109750 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd9e00700 1 -- 172.21.15.12:0/1666671340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 msgr2=0x7f8bd410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/1666671340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40098a0 tx=0x7f8bc4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd9e00700 1 -- 172.21.15.12:0/1666671340 shutdown_connections 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/1666671340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd410bb40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.190+0000 7f8bd9e00700 1 -- 172.21.15.12:0/1666671340 >> 172.21.15.12:0/1666671340 conn(0x7f8bd41031a0 msgr2=0x7f8bd41055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.191+0000 7f8bd9e00700 1 -- 172.21.15.12:0/1666671340 shutdown_connections 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.191+0000 7f8bd9e00700 1 -- 172.21.15.12:0/1666671340 wait complete. 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.192+0000 7f8bd9e00700 1 Processor -- start 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.192+0000 7f8bd9e00700 1 -- start start 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.192+0000 7f8bd9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.192+0000 7f8bd9e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd410d090 con 0x7f8bd4109750 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.193+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.193+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39728/0 (socket says 172.21.15.12:39728) 2024-04-26T21:51:32.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.193+0000 7f8bd2a00700 1 -- 172.21.15.12:0/3360147295 learned_addr learned my addr 172.21.15.12:0/3360147295 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.193+0000 7f8bd2a00700 1 -- 172.21.15.12:0/3360147295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd419f2f0 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.193+0000 7f8bd2a00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40116e0 tx=0x7f8bc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.194+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc4003a60 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.194+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc401b8f0 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.194+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd419f580 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.194+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc4019860 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.194+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a2230 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.195+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4064a80 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.195+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8bc4019b10 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.196+0000 7f8bd8a00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 0x7f8bb803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.196+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8bc404dd50 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.196+0000 7f8bca000700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 0x7f8bb803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.197+0000 7f8bca000700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 0x7f8bb803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.202+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8bc401f030 con 0x7f8bd4109750 2024-04-26T21:51:32.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.436+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8bd41a2400 con 0x7f8bd4109750 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.441+0000 7f8bd8a00700 1 -- 172.21.15.12:0/3360147295 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8bd41a2400 con 0x7f8bd4109750 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 msgr2=0x7f8bb803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 0x7f8bb803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 msgr2=0x7f8bd419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40116e0 tx=0x7f8bc4011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 shutdown_connections 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f8bb80395b0 0x7f8bb803ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 --2- 172.21.15.12:0/3360147295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bd4109750 0x7f8bd419edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 >> 172.21.15.12:0/3360147295 conn(0x7f8bd41031a0 msgr2=0x7f8bd4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 shutdown_connections 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:31.444+0000 7f8bd9e00700 1 -- 172.21.15.12:0/3360147295 wait complete. 2024-04-26T21:51:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-mon[28184]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-mon[28184]: from='mgr.14120 172.21.15.12:0/4178795545' entity='mgr.x' 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2718296342' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setuser ceph since I am not root 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setgroup ceph since I am not root 2024-04-26T21:51:33.995 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:33.993+0000 7f6ddea2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:51:34.517 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:34.259+0000 7f6ddea2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:51:34.517 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:34.397+0000 7f6ddea2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.733+0000 7f71cfc00700 1 Processor -- start 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.734+0000 7f71cfc00700 1 -- start start 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.734+0000 7f71cfc00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.734+0000 7f71cfc00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71c807d8c0 con 0x7f71c807ee90 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.734+0000 7f71cd200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.735+0000 7f71cd200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39742/0 (socket says 172.21.15.12:39742) 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.735+0000 7f71cd200700 1 -- 172.21.15.12:0/1656911362 learned_addr learned my addr 172.21.15.12:0/1656911362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.735+0000 7f71cd200700 1 -- 172.21.15.12:0/1656911362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c807da40 con 0x7f71c807ee90 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.735+0000 7f71cd200700 1 --2- 172.21.15.12:0/1656911362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0098a0 tx=0x7f71bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e991f96936d75a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.736+0000 7f71c7400700 1 -- 172.21.15.12:0/1656911362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71bc004160 con 0x7f71c807ee90 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.736+0000 7f71c7400700 1 -- 172.21.15.12:0/1656911362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc004300 con 0x7f71c807ee90 2024-04-26T21:51:34.805 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.736+0000 7f71c7400700 1 -- 172.21.15.12:0/1656911362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71bc011a20 con 0x7f71c807ee90 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 -- 172.21.15.12:0/1656911362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 msgr2=0x7f71c807d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 --2- 172.21.15.12:0/1656911362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0098a0 tx=0x7f71bc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 -- 172.21.15.12:0/1656911362 shutdown_connections 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 --2- 172.21.15.12:0/1656911362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c807d2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 -- 172.21.15.12:0/1656911362 >> 172.21.15.12:0/1656911362 conn(0x7f71c8104290 msgr2=0x7f71c81066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 -- 172.21.15.12:0/1656911362 shutdown_connections 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.737+0000 7f71cfc00700 1 -- 172.21.15.12:0/1656911362 wait complete. 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.738+0000 7f71cfc00700 1 Processor -- start 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.738+0000 7f71cfc00700 1 -- start start 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.739+0000 7f71cfc00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.739+0000 7f71cfc00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71c810f310 con 0x7f71c807ee90 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.739+0000 7f71cd200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.739+0000 7f71cd200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39752/0 (socket says 172.21.15.12:39752) 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.739+0000 7f71cd200700 1 -- 172.21.15.12:0/2718296342 learned_addr learned my addr 172.21.15.12:0/2718296342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71cd200700 1 -- 172.21.15.12:0/2718296342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c819f350 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71cd200700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f71bc004660 tx=0x7f71bc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71bc003aa0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc01b8f0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71bc023900 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.740+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c819f5e0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.741+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a22d0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.742+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c8064a80 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.742+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f71bc01baa0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.743+0000 7f71c5600700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 0x7f71b003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.743+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71bc04def0 con 0x7f71c807ee90 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.743+0000 7f71c7e00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 0x7f71b003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.744+0000 7f71c7e00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 0x7f71b003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71b4006fd0 tx=0x7f71b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:32.749+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f71bc01f030 con 0x7f71c807ee90 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.045+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f71c81a24a0 con 0x7f71c807ee90 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.664+0000 7f71c5600700 1 -- 172.21.15.12:0/2718296342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f71bc04d840 con 0x7f71c807ee90 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.669+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 msgr2=0x7f71b003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.669+0000 7f71cfc00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 0x7f71b003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71b4006fd0 tx=0x7f71b4006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.669+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 msgr2=0x7f71c819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f71bc004660 tx=0x7f71bc011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 shutdown_connections 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f71b0039600 0x7f71b003bac0 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71b4006fd0 tx=0x7f71b4006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 --2- 172.21.15.12:0/2718296342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71c807ee90 0x7f71c819ee10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 >> 172.21.15.12:0/2718296342 conn(0x7f71c8104290 msgr2=0x7f71c8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:34.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 shutdown_connections 2024-04-26T21:51:34.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:33.670+0000 7f71cfc00700 1 -- 172.21.15.12:0/2718296342 wait complete. 2024-04-26T21:51:34.927 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:34.807+0000 7f6ddea2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:51:34.928 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:34.927+0000 7f6ddea2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:51:34.928 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2718296342' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T21:51:34.928 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:34 smithi012 ceph-mon[28184]: mgrmap e9: x(active, since 25s) 2024-04-26T21:51:36.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:36 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3536246703' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:51:36.790 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:36.568+0000 7f6ddea2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:51:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:51:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T21:51:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:51:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:51:36.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.713+0000 7f7a95e00700 1 Processor -- start 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.713+0000 7f7a95e00700 1 -- start start 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.713+0000 7f7a95e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.713+0000 7f7a95e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901096b0 con 0x7f7a90108cc0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.713+0000 7f7a8ea00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.714+0000 7f7a8ea00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39760/0 (socket says 172.21.15.12:39760) 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.714+0000 7f7a8ea00700 1 -- 172.21.15.12:0/1244378680 learned_addr learned my addr 172.21.15.12:0/1244378680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.714+0000 7f7a8ea00700 1 -- 172.21.15.12:0/1244378680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a90109ec0 con 0x7f7a90108cc0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.714+0000 7f7a8ea00700 1 --2- 172.21.15.12:0/1244378680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a800098a0 tx=0x7f7a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73c3d27eab8826b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.714+0000 7f7a8d600700 1 -- 172.21.15.12:0/1244378680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a80004160 con 0x7f7a90108cc0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.715+0000 7f7a8d600700 1 -- 172.21.15.12:0/1244378680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a80004300 con 0x7f7a90108cc0 2024-04-26T21:51:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.715+0000 7f7a8d600700 1 -- 172.21.15.12:0/1244378680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a80011a20 con 0x7f7a90108cc0 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.715+0000 7f7a95e00700 1 -- 172.21.15.12:0/1244378680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 msgr2=0x7f7a901090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.715+0000 7f7a95e00700 1 --2- 172.21.15.12:0/1244378680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a800098a0 tx=0x7f7a80009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.716+0000 7f7a95e00700 1 -- 172.21.15.12:0/1244378680 shutdown_connections 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.716+0000 7f7a95e00700 1 --2- 172.21.15.12:0/1244378680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a901090e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.716+0000 7f7a95e00700 1 -- 172.21.15.12:0/1244378680 >> 172.21.15.12:0/1244378680 conn(0x7f7a901042a0 msgr2=0x7f7a901066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.716+0000 7f7a95e00700 1 -- 172.21.15.12:0/1244378680 shutdown_connections 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.716+0000 7f7a95e00700 1 -- 172.21.15.12:0/1244378680 wait complete. 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.717+0000 7f7a95e00700 1 Processor -- start 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a95e00700 1 -- start start 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a95e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a95e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010d270 con 0x7f7a90108cc0 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a8ea00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a8ea00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39776/0 (socket says 172.21.15.12:39776) 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.718+0000 7f7a8ea00700 1 -- 172.21.15.12:0/3536246703 learned_addr learned my addr 172.21.15.12:0/3536246703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.719+0000 7f7a8ea00700 1 -- 172.21.15.12:0/3536246703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9007f0c0 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.719+0000 7f7a8ea00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a800116e0 tx=0x7f7a80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.719+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a800039e0 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.720+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8001b8f0 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.720+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9007d2b0 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.720+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a80023760 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.720+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9007d7f0 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.721+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a90064a80 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.721+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7a8001b440 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.721+0000 7f7a94a00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 0x7f7a7403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.722+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a8004df50 con 0x7f7a90108cc0 2024-04-26T21:51:36.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.722+0000 7f7a8e000700 1 -- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 msgr2=0x7f7a7403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.722+0000 7f7a8e000700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 0x7f7a7403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.727+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7a8001f030 con 0x7f7a90108cc0 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.923+0000 7f7a8e000700 1 -- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 msgr2=0x7f7a7403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:35.923+0000 7f7a8e000700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 0x7f7a7403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.017+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7a9007e090 con 0x7f7a90108cc0 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.018+0000 7f7a94a00700 1 -- 172.21.15.12:0/3536246703 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7a8000e260 con 0x7f7a90108cc0 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 msgr2=0x7f7a7403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 0x7f7a7403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 msgr2=0x7f7a9007eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a800116e0 tx=0x7f7a80011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 shutdown_connections 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f7a740395b0 0x7f7a7403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 --2- 172.21.15.12:0/3536246703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a90108cc0 0x7f7a9007eb80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 >> 172.21.15.12:0/3536246703 conn(0x7f7a901042a0 msgr2=0x7f7a90104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:36.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.021+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 shutdown_connections 2024-04-26T21:51:36.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:36.022+0000 7f7a95e00700 1 -- 172.21.15.12:0/3536246703 wait complete. 2024-04-26T21:51:36.811 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-04-26T21:51:36.811 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-04-26T21:51:37.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:36.804+0000 7f6ddea2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:51:37.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:37.030+0000 7f6ddea2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:51:37.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:37.434+0000 7f6ddea2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:51:37.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:37.560+0000 7f6ddea2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:51:38.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:37.822+0000 7f6ddea2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:51:38.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:38 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:38.086+0000 7f6ddea2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:51:39.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:38 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:38.904+0000 7f6ddea2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:51:39.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:39.472+0000 7f6ddea2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:51:39.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:39.606+0000 7f6ddea2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:51:40.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:39.883+0000 7f6ddea2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:51:40.683 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:40.554+0000 7f6ddea2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:51:41.015 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:40.683+0000 7f6ddea2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:51:41.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:41 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:41.014+0000 7f6ddea2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:51:41.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:41 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:41.198+0000 7f6ddea2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:51:41.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:41 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:41.331+0000 7f6ddea2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:51:41.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:41 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:41.490+0000 7f6ddea2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:51:42.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:41 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:41.869+0000 7f6ddea2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:51:42.976 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:42.668+0000 7f6ddea2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 21:51:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T21:51:42.975+0000 7f6ddea2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: Active manager daemon x restarted 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: Activating manager daemon x 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: mgrmap e10: x(active, starting, since 0.00525547s) 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:51:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:51:43.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:51:43.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:51:43.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:51:43.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:43 smithi012 ceph-mon[28184]: Manager daemon x is now available 2024-04-26T21:51:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:51:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:51:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:44 smithi012 ceph-mon[28184]: mgrmap e11: x(active, since 1.00864s) 2024-04-26T21:51:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:51:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:51:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:51:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:51:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.402+0000 7f3850c00700 1 Processor -- start 2024-04-26T21:51:44.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.403+0000 7f3850c00700 1 -- start start 2024-04-26T21:51:44.788 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.403+0000 7f3850c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.403+0000 7f3850c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f384c07ce80 con 0x7f384c109790 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.404+0000 7f3849e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.404+0000 7f3849e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39786/0 (socket says 172.21.15.12:39786) 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.404+0000 7f3849e00700 1 -- 172.21.15.12:0/120543658 learned_addr learned my addr 172.21.15.12:0/120543658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.404+0000 7f3849e00700 1 -- 172.21.15.12:0/120543658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c10c0c0 con 0x7f384c109790 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.405+0000 7f3849e00700 1 --2- 172.21.15.12:0/120543658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f383c0098a0 tx=0x7f383c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2865fc55ddd8a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.405+0000 7f3848a00700 1 -- 172.21.15.12:0/120543658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f383c004160 con 0x7f384c109790 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.405+0000 7f3848a00700 1 -- 172.21.15.12:0/120543658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c004300 con 0x7f384c109790 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.405+0000 7f3848a00700 1 -- 172.21.15.12:0/120543658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f383c011a20 con 0x7f384c109790 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 -- 172.21.15.12:0/120543658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 msgr2=0x7f384c10bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 --2- 172.21.15.12:0/120543658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f383c0098a0 tx=0x7f383c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 -- 172.21.15.12:0/120543658 shutdown_connections 2024-04-26T21:51:44.789 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 --2- 172.21.15.12:0/120543658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c10bb80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 -- 172.21.15.12:0/120543658 >> 172.21.15.12:0/120543658 conn(0x7f384c1031a0 msgr2=0x7f384c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:44.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 -- 172.21.15.12:0/120543658 shutdown_connections 2024-04-26T21:51:44.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.406+0000 7f3850c00700 1 -- 172.21.15.12:0/120543658 wait complete. 2024-04-26T21:51:44.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.407+0000 7f3850c00700 1 Processor -- start 2024-04-26T21:51:44.790 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3850c00700 1 -- start start 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3850c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3850c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f384c10d0c0 con 0x7f384c109790 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3849e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3849e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39802/0 (socket says 172.21.15.12:39802) 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.408+0000 7f3849e00700 1 -- 172.21.15.12:0/2116569931 learned_addr learned my addr 172.21.15.12:0/2116569931 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3849e00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c109550 con 0x7f384c109790 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3849e00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f383c000c00 tx=0x7f383c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f383c003aa0 con 0x7f384c109790 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c01b8f0 con 0x7f384c109790 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f383c023870 con 0x7f384c109790 2024-04-26T21:51:44.791 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c107740 con 0x7f384c109790 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.409+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c107c80 con 0x7f384c109790 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f383c023b70 con 0x7f384c109790 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3842a00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f383003c1d0 con 0x7f3830039600 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f383c04dfa0 con 0x7f384c109790 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3849400700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.411+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.612+0000 7f3849400700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:37.612+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:38.013+0000 7f3849400700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:38.013+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:38.814+0000 7f3849400700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:44.792 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:38.814+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:40.416+0000 7f3849400700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2513368485 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:40.416+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:42.982+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f383c01a3d0 con 0x7f384c109790 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:42.982+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 msgr2=0x7f383003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:42.982+0000 7f3842a00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.984+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f383c00bce0 con 0x7f384c109790 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.985+0000 7f3842a00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 0x7f383003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.985+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f383c00e9b0 con 0x7f383003cfe0 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.985+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 0x7f383003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.986+0000 7f3849400700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 0x7f383003f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3838003a10 tx=0x7f3838009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:43.987+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f383c00e9b0 con 0x7f383003cfe0 2024-04-26T21:51:44.793 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.000+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f384c064a80 con 0x7f383003cfe0 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3842a00700 1 -- 172.21.15.12:0/2116569931 <== mgr.14152 v2:172.21.15.12:6800/3660687894 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f384c064a80 con 0x7f383003cfe0 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 msgr2=0x7f383003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 0x7f383003f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3838003a10 tx=0x7f3838009270 comp rx=0 tx=0).stop 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 msgr2=0x7f384c109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f383c000c00 tx=0x7f383c011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 shutdown_connections 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f383003cfe0 0x7f383003f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f384c109790 0x7f384c109010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 --2- 172.21.15.12:0/2116569931 >> [v2:172.21.15.12:6800/2513368485,v1:172.21.15.12:6801/2513368485] conn(0x7f3830039600 0x7f383003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 >> 172.21.15.12:0/2116569931 conn(0x7f384c1031a0 msgr2=0x7f384c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.001+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 shutdown_connections 2024-04-26T21:51:44.794 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:44.002+0000 7f3850c00700 1 -- 172.21.15.12:0/2116569931 wait complete. 2024-04-26T21:51:44.795 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-04-26T21:51:44.795 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:51:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:45 smithi012 ceph-mon[28184]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:51:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:45 smithi012 ceph-mon[28184]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:51:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:44] ENGINE Bus STARTING 2024-04-26T21:51:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:45] ENGINE Serving on http://172.21.15.12:8765 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: mgrmap e12: x(active, since 2s) 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:45] ENGINE Serving on https://172.21.15.12:7150 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: [26/Apr/2024:21:51:45] ENGINE Bus STARTED 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.527+0000 7f1abf000700 1 Processor -- start 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.527+0000 7f1abf000700 1 -- start start 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.527+0000 7f1abf000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.527+0000 7f1abf000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8083560 con 0x7f1ab810fce0 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.528+0000 7f1ab7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.528+0000 7f1ab7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41390/0 (socket says 172.21.15.12:41390) 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.528+0000 7f1ab7e00700 1 -- 172.21.15.12:0/1552525406 learned_addr learned my addr 172.21.15.12:0/1552525406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:46.959 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.528+0000 7f1ab7e00700 1 -- 172.21.15.12:0/1552525406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab8112610 con 0x7f1ab810fce0 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.529+0000 7f1ab7e00700 1 --2- 172.21.15.12:0/1552525406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098a0 tx=0x7f1aa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c4d819b97c2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.529+0000 7f1ab7400700 1 -- 172.21.15.12:0/1552525406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8004160 con 0x7f1ab810fce0 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.529+0000 7f1ab7400700 1 -- 172.21.15.12:0/1552525406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa8004300 con 0x7f1ab810fce0 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.529+0000 7f1ab7400700 1 -- 172.21.15.12:0/1552525406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8011a20 con 0x7f1ab810fce0 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 -- 172.21.15.12:0/1552525406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 msgr2=0x7f1ab81120d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 --2- 172.21.15.12:0/1552525406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098a0 tx=0x7f1aa8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 -- 172.21.15.12:0/1552525406 shutdown_connections 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 --2- 172.21.15.12:0/1552525406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81120d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.960 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 -- 172.21.15.12:0/1552525406 >> 172.21.15.12:0/1552525406 conn(0x7f1ab807e2e0 msgr2=0x7f1ab8080700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 -- 172.21.15.12:0/1552525406 shutdown_connections 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.530+0000 7f1abf000700 1 -- 172.21.15.12:0/1552525406 wait complete. 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.531+0000 7f1abf000700 1 Processor -- start 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1abf000700 1 -- start start 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1abf000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1abf000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81136d0 con 0x7f1ab810fce0 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1ab7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1ab7e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41396/0 (socket says 172.21.15.12:41396) 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.532+0000 7f1ab7e00700 1 -- 172.21.15.12:0/2277517126 learned_addr learned my addr 172.21.15.12:0/2277517126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.533+0000 7f1ab7e00700 1 -- 172.21.15.12:0/2277517126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81a3740 con 0x7f1ab810fce0 2024-04-26T21:51:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1ab7e00700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004660 tx=0x7f1aa8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8003aa0 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa801b8f0 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab81a39d0 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa8023900 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.534+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81a66c0 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.535+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f1aa8023bc0 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.535+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8064a80 con 0x7f1ab810fce0 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.536+0000 7f1ab5600700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 0x7f1a9c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:46.962 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.536+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1aa804e0e0 con 0x7f1ab810fce0 2024-04-26T21:51:46.964 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.536+0000 7f1aaf400700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 0x7f1a9c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.537+0000 7f1aaf400700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 0x7f1a9c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000ad30 tx=0x7f1aa00093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.543+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1aa801f030 con 0x7f1ab810fce0 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.788+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1ab807f4b0 con 0x7f1a9c039600 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.928+0000 7f1ab5600700 1 -- 172.21.15.12:0/2277517126 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1ab807f4b0 con 0x7f1a9c039600 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 msgr2=0x7f1a9c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 0x7f1a9c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000ad30 tx=0x7f1aa00093f0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 msgr2=0x7f1ab81a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004660 tx=0x7f1aa8011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 shutdown_connections 2024-04-26T21:51:46.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a9c039600 0x7f1a9c03bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 --2- 172.21.15.12:0/2277517126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ab810fce0 0x7f1ab81a3200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:46.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 >> 172.21.15.12:0/2277517126 conn(0x7f1ab807e2e0 msgr2=0x7f1ab807eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:46.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.931+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 shutdown_connections 2024-04-26T21:51:46.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:45.932+0000 7f1abf000700 1 -- 172.21.15.12:0/2277517126 wait complete. 2024-04-26T21:51:46.966 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-04-26T21:51:48.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:49.040 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ab9k2QxBzznuozZyGaITp.L09TdX79oEmOygRK81e7Ffs.y7H6Gbe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714168308, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.626+0000 7efc52200700 1 Processor -- start 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.626+0000 7efc52200700 1 -- start start 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.626+0000 7efc52200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.626+0000 7efc52200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c07ce00 con 0x7efc4c109810 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.627+0000 7efc4b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.627+0000 7efc4b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41400/0 (socket says 172.21.15.12:41400) 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.627+0000 7efc4b400700 1 -- 172.21.15.12:0/1372502064 learned_addr learned my addr 172.21.15.12:0/1372502064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.627+0000 7efc4b400700 1 -- 172.21.15.12:0/1372502064 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c10c140 con 0x7efc4c109810 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.628+0000 7efc4b400700 1 --2- 172.21.15.12:0/1372502064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009e80 tx=0x7efc3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4032f95e2458779f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.628+0000 7efc4aa00700 1 -- 172.21.15.12:0/1372502064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc4c109810 2024-04-26T21:51:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.628+0000 7efc4aa00700 1 -- 172.21.15.12:0/1372502064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3c00b8c0 con 0x7efc4c109810 2024-04-26T21:51:49.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.628+0000 7efc4aa00700 1 -- 172.21.15.12:0/1372502064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc3c004490 con 0x7efc4c109810 2024-04-26T21:51:49.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 -- 172.21.15.12:0/1372502064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 msgr2=0x7efc4c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:49.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 --2- 172.21.15.12:0/1372502064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009e80 tx=0x7efc3c00b330 comp rx=0 tx=0).stop 2024-04-26T21:51:49.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 -- 172.21.15.12:0/1372502064 shutdown_connections 2024-04-26T21:51:49.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 --2- 172.21.15.12:0/1372502064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c10bc00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:49.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 -- 172.21.15.12:0/1372502064 >> 172.21.15.12:0/1372502064 conn(0x7efc4c1031a0 msgr2=0x7efc4c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:49.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 -- 172.21.15.12:0/1372502064 shutdown_connections 2024-04-26T21:51:49.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.629+0000 7efc52200700 1 -- 172.21.15.12:0/1372502064 wait complete. 2024-04-26T21:51:49.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.630+0000 7efc52200700 1 Processor -- start 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc52200700 1 -- start start 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc52200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc52200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c10d090 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc4b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc4b400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41414/0 (socket says 172.21.15.12:41414) 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.631+0000 7efc4b400700 1 -- 172.21.15.12:0/4183272984 learned_addr learned my addr 172.21.15.12:0/4183272984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.632+0000 7efc4b400700 1 -- 172.21.15.12:0/4183272984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c19d0f0 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.632+0000 7efc4b400700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc3c004220 tx=0x7efc3c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.632+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc3c020020 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.633+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3c003b50 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.633+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c19d380 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.633+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc3c017040 con 0x7efc4c109810 2024-04-26T21:51:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.633+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a0070 con 0x7efc4c109810 2024-04-26T21:51:49.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.634+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c064a80 con 0x7efc4c109810 2024-04-26T21:51:49.074 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.634+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7efc3c023020 con 0x7efc4c109810 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.635+0000 7efc48c00700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 0x7efc3003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.635+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efc3c00f050 con 0x7efc4c109810 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.635+0000 7efc42a00700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 0x7efc3003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.636+0000 7efc42a00700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 0x7efc3003bb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc34006fd0 tx=0x7efc34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.641+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc3c02a020 con 0x7efc4c109810 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:47.892+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7efc4c104330 con 0x7efc30039650 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.177+0000 7efc48c00700 1 -- 172.21.15.12:0/4183272984 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7efc4c104330 con 0x7efc30039650 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.180+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 msgr2=0x7efc3003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.180+0000 7efc52200700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 0x7efc3003bb10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc34006fd0 tx=0x7efc34006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.180+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 msgr2=0x7efc4c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.180+0000 7efc52200700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc3c004220 tx=0x7efc3c0093e0 comp rx=0 tx=0).stop 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.180+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 shutdown_connections 2024-04-26T21:51:49.075 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.181+0000 7efc52200700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efc30039650 0x7efc3003bb10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:49.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.181+0000 7efc52200700 1 --2- 172.21.15.12:0/4183272984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c109810 0x7efc4c19cbb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:49.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.181+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 >> 172.21.15.12:0/4183272984 conn(0x7efc4c1031a0 msgr2=0x7efc4c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:49.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.181+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 shutdown_connections 2024-04-26T21:51:49.076 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:48.181+0000 7efc52200700 1 -- 172.21.15.12:0/4183272984 wait complete. 2024-04-26T21:51:49.076 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-04-26T21:51:49.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:49 smithi012 ceph-mon[28184]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:49.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:49 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:49.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:49 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:50.275 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:50 smithi012 ceph-mon[28184]: mgrmap e13: x(active, since 6s) 2024-04-26T21:51:50.275 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/831033408' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:51:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.704+0000 7f1bd3e00700 1 Processor -- start 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.704+0000 7f1bd3e00700 1 -- start start 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.705+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.705+0000 7f1bd3e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd41094a0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.706+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.706+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54154/0 (socket says 172.21.15.12:54154) 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.706+0000 7f1bd2a00700 1 -- 172.21.15.12:0/2512993352 learned_addr learned my addr 172.21.15.12:0/2512993352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.706+0000 7f1bd2a00700 1 -- 172.21.15.12:0/2512993352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd4109cb0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.707+0000 7f1bd2a00700 1 --2- 172.21.15.12:0/2512993352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40098a0 tx=0x7f1bc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565c8983511f69b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.707+0000 7f1bd1600700 1 -- 172.21.15.12:0/2512993352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bc4004160 con 0x7f1bd4108ab0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.707+0000 7f1bd1600700 1 -- 172.21.15.12:0/2512993352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc4004300 con 0x7f1bd4108ab0 2024-04-26T21:51:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.707+0000 7f1bd1600700 1 -- 172.21.15.12:0/2512993352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bc4011a20 con 0x7f1bd4108ab0 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 -- 172.21.15.12:0/2512993352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 msgr2=0x7f1bd4108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/2512993352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40098a0 tx=0x7f1bc4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 -- 172.21.15.12:0/2512993352 shutdown_connections 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/2512993352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd4108ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 -- 172.21.15.12:0/2512993352 >> 172.21.15.12:0/2512993352 conn(0x7f1bd4104050 msgr2=0x7f1bd4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 -- 172.21.15.12:0/2512993352 shutdown_connections 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.708+0000 7f1bd3e00700 1 -- 172.21.15.12:0/2512993352 wait complete. 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.709+0000 7f1bd3e00700 1 Processor -- start 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd3e00700 1 -- start start 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd3e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410d060 con 0x7f1bd4108ab0 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54158/0 (socket says 172.21.15.12:54158) 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.710+0000 7f1bd2a00700 1 -- 172.21.15.12:0/831033408 learned_addr learned my addr 172.21.15.12:0/831033408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.711+0000 7f1bd2a00700 1 -- 172.21.15.12:0/831033408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd407f0c0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.711+0000 7f1bd2a00700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009520 tx=0x7f1bc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.711+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bc40039e0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.712+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc401b8f0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.712+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd407d2b0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.712+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bc4019770 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.712+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd407d7f0 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.713+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1bc4019a20 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.713+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd4064a80 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.713+0000 7f1bcb400700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 0x7f1bb803bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.714+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1bc404e190 con 0x7f1bd4108ab0 2024-04-26T21:51:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.714+0000 7f1bd2000700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 0x7f1bb803bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.715+0000 7f1bd2000700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 0x7f1bb803bbb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0006fd0 tx=0x7f1bc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.719+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1bc401f030 con 0x7f1bd4108ab0 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.960+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1bd407df90 con 0x7f1bd4108ab0 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.960+0000 7f1bcb400700 1 -- 172.21.15.12:0/831033408 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1bc4019d10 con 0x7f1bd4108ab0 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 msgr2=0x7f1bb803bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 0x7f1bb803bbb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0006fd0 tx=0x7f1bc0006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 msgr2=0x7f1bd407eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009520 tx=0x7f1bc4011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 shutdown_connections 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bb80396f0 0x7f1bb803bbb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 --2- 172.21.15.12:0/831033408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bd4108ab0 0x7f1bd407eb80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.963+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 >> 172.21.15.12:0/831033408 conn(0x7f1bd4104050 msgr2=0x7f1bd4104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.964+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 shutdown_connections 2024-04-26T21:51:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:49.964+0000 7f1bd3e00700 1 -- 172.21.15.12:0/831033408 wait complete. 2024-04-26T21:51:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:51:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-26T21:51:50.637 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:51:50.637 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-26T21:51:50.638 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-26T21:51:50.638 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 0ju9868h83 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:50.642 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config directory 2024-04-26T21:51:50.643 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:51:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.063+0000 7fe3f6000700 1 Processor -- start 2024-04-26T21:51:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.063+0000 7fe3f6000700 1 -- start start 2024-04-26T21:51:53.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.063+0000 7fe3f6000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:53.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.063+0000 7fe3f6000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01096b0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.064+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:53.124 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.064+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54166/0 (socket says 172.21.15.12:54166) 2024-04-26T21:51:53.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.064+0000 7fe3ef400700 1 -- 172.21.15.12:0/3144428058 learned_addr learned my addr 172.21.15.12:0/3144428058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:53.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.064+0000 7fe3ef400700 1 -- 172.21.15.12:0/3144428058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f0109ec0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.065+0000 7fe3ef400700 1 --2- 172.21.15.12:0/3144428058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00098a0 tx=0x7fe3e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f8eede1a5778c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:53.127 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.065+0000 7fe3ee000700 1 -- 172.21.15.12:0/3144428058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e0004160 con 0x7fe3f0108cc0 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.065+0000 7fe3ee000700 1 -- 172.21.15.12:0/3144428058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e0004300 con 0x7fe3f0108cc0 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.066+0000 7fe3ee000700 1 -- 172.21.15.12:0/3144428058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e0011a20 con 0x7fe3f0108cc0 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.066+0000 7fe3f6000700 1 -- 172.21.15.12:0/3144428058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 msgr2=0x7fe3f01090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.066+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3144428058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00098a0 tx=0x7fe3e0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.067+0000 7fe3f6000700 1 -- 172.21.15.12:0/3144428058 shutdown_connections 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.067+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3144428058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f01090e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.067+0000 7fe3f6000700 1 -- 172.21.15.12:0/3144428058 >> 172.21.15.12:0/3144428058 conn(0x7fe3f01042a0 msgr2=0x7fe3f01066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.067+0000 7fe3f6000700 1 -- 172.21.15.12:0/3144428058 shutdown_connections 2024-04-26T21:51:53.128 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.067+0000 7fe3f6000700 1 -- 172.21.15.12:0/3144428058 wait complete. 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.068+0000 7fe3f6000700 1 Processor -- start 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.068+0000 7fe3f6000700 1 -- start start 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.068+0000 7fe3f6000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.068+0000 7fe3f6000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010d270 con 0x7fe3f0108cc0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.069+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.069+0000 7fe3ef400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54168/0 (socket says 172.21.15.12:54168) 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.069+0000 7fe3ef400700 1 -- 172.21.15.12:0/3092020634 learned_addr learned my addr 172.21.15.12:0/3092020634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.069+0000 7fe3ef400700 1 -- 172.21.15.12:0/3092020634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f019d0f0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.069+0000 7fe3ef400700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009520 tx=0x7fe3e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.070+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e00039e0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.070+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e001b8f0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.070+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3e0019770 con 0x7fe3f0108cc0 2024-04-26T21:51:53.129 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.070+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f019d380 con 0x7fe3f0108cc0 2024-04-26T21:51:53.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.070+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f01a0070 con 0x7fe3f0108cc0 2024-04-26T21:51:53.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.071+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe3e001b440 con 0x7fe3f0108cc0 2024-04-26T21:51:53.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.071+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f0064a80 con 0x7fe3f0108cc0 2024-04-26T21:51:53.130 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.071+0000 7fe3e7e00700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 0x7fe3d403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.072+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe3e004e160 con 0x7fe3f0108cc0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.072+0000 7fe3eea00700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 0x7fe3d403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.072+0000 7fe3eea00700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 0x7fe3d403bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006fd0 tx=0x7fe3dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.077+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe3e001f030 con 0x7fe3f0108cc0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.314+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe3f01a06d0 con 0x7fe3f0108cc0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.315+0000 7fe3e7e00700 1 -- 172.21.15.12:0/3092020634 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe3e0050160 con 0x7fe3f0108cc0 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.317+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 msgr2=0x7fe3d403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.317+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 0x7fe3d403bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006fd0 tx=0x7fe3dc006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.317+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 msgr2=0x7fe3f019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.317+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009520 tx=0x7fe3e0011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 shutdown_connections 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe3d40396f0 0x7fe3d403bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 --2- 172.21.15.12:0/3092020634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3f0108cc0 0x7fe3f019cbb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:53.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 >> 172.21.15.12:0/3092020634 conn(0x7fe3f01042a0 msgr2=0x7fe3f0104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:53.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 shutdown_connections 2024-04-26T21:51:53.133 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:52.318+0000 7fe3f6000700 1 -- 172.21.15.12:0/3092020634 wait complete. 2024-04-26T21:51:54.735 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.703+0000 7f3c15800700 1 Processor -- start 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.703+0000 7f3c15800700 1 -- start start 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.703+0000 7f3c15800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.703+0000 7f3c15800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1010b6f0 con 0x7f3c1010ad00 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.704+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.704+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54176/0 (socket says 172.21.15.12:54176) 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.704+0000 7f3c0fe00700 1 -- 172.21.15.12:0/4105700284 learned_addr learned my addr 172.21.15.12:0/4105700284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.704+0000 7f3c0fe00700 1 -- 172.21.15.12:0/4105700284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1010bf00 con 0x7f3c1010ad00 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.705+0000 7f3c0fe00700 1 --2- 172.21.15.12:0/4105700284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098a0 tx=0x7f3c00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3008309c133e95a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.705+0000 7f3c0ea00700 1 -- 172.21.15.12:0/4105700284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00004160 con 0x7f3c1010ad00 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.705+0000 7f3c0ea00700 1 -- 172.21.15.12:0/4105700284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00004300 con 0x7f3c1010ad00 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.705+0000 7f3c0ea00700 1 -- 172.21.15.12:0/4105700284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00011a20 con 0x7f3c1010ad00 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 -- 172.21.15.12:0/4105700284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 msgr2=0x7f3c1010b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:54.736 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 --2- 172.21.15.12:0/4105700284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098a0 tx=0x7f3c00009bb0 comp rx=0 tx=0).stop 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 -- 172.21.15.12:0/4105700284 shutdown_connections 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 --2- 172.21.15.12:0/4105700284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1010b120 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 -- 172.21.15.12:0/4105700284 >> 172.21.15.12:0/4105700284 conn(0x7f3c1007e0e0 msgr2=0x7f3c1007e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 -- 172.21.15.12:0/4105700284 shutdown_connections 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.706+0000 7f3c15800700 1 -- 172.21.15.12:0/4105700284 wait complete. 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.707+0000 7f3c15800700 1 Processor -- start 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.707+0000 7f3c15800700 1 -- start start 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c15800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c15800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1010f2b0 con 0x7f3c1010ad00 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54186/0 (socket says 172.21.15.12:54186) 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c0fe00700 1 -- 172.21.15.12:0/1371860176 learned_addr learned my addr 172.21.15.12:0/1371860176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.708+0000 7f3c0fe00700 1 -- 172.21.15.12:0/1371860176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1019f160 con 0x7f3c1010ad00 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c0fe00700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009520 tx=0x7f3c00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:54.737 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c000039e0 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0001b8f0 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00019770 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1019f3f0 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.709+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a20e0 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.710+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c10064a80 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.710+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c0001b440 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.711+0000 7f3c0cc00700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 0x7f3bf403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.711+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c0004e130 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.711+0000 7f3c0f400700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 0x7f3bf403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.712+0000 7f3c0f400700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 0x7f3bf403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:53.716+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3c0001f030 con 0x7f3c1010ad00 2024-04-26T21:51:54.738 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.071+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3c101a26d0 con 0x7f3c1010ad00 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.077+0000 7f3c0cc00700 1 -- 172.21.15.12:0/1371860176 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3c0000e260 con 0x7f3c1010ad00 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.079+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 msgr2=0x7f3bf403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.079+0000 7f3c15800700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 0x7f3bf403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.079+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 msgr2=0x7f3c1019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.079+0000 7f3c15800700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009520 tx=0x7f3c00011aa0 comp rx=0 tx=0).stop 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 shutdown_connections 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40396f0 0x7f3bf403bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 --2- 172.21.15.12:0/1371860176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c1010ad00 0x7f3c1019ec20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 >> 172.21.15.12:0/1371860176 conn(0x7f3c1007e0e0 msgr2=0x7f3c10108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 shutdown_connections 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-26T21:51:54.080+0000 7f3c15800700 1 -- 172.21.15.12:0/1371860176 wait complete. 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:51:54.740 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-04-26T21:51:54.817 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:51:54.817 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:51:54.818 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:51:54.866 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:51:54.866 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:51:54.866 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:51:54.948 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:51:54.948 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:51:54.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:51:55.086 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:51:55.087 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:51:55.087 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:51:55.140 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:51:55.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:51:55.244 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:51:55.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:55 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1371860176' entity='client.admin' 2024-04-26T21:51:55.274 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:51:55.313 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:51:55.325 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:51:55.364 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWomrY/aLzUlkEj5aw+1Og7qmfTjiE4WLjNOKaGBbRb/KQROR+ueFWGfXl4G8FdzIoAdXB6+OU0xS2ogYFL3gp6cSrMxYtclVA6aqQmuYKoX8rUKsGtXpeZL8utZoD3jUzK7Tx2V0bRGnII3DGVjT/joh+dtHvjyrBoAuZprW8FoLQrNiRJOWrAufj+urv2TmYPztEFQPulpX33oeAdgoRdCWo6zbXGt+evBO+EeRXdhD2540kXFJaWeTJ+lZbdoDYWaTFGHW7hsQFeFCFO446sXAxtQvnIbMCeuo37ucZMah0xZfLRpbgwNVQ0gb3SxXmWBhXcuna6snxLwB7eoe8PGTheTTq3Jv04Ex6dG7hf5Z0U+M5yEaA6Y897h3ufLcGC1CzWmXObcocdZjLvL6JCl5VnRiGAgYIR7YxuKFnNoltSYh7w3oGA7INVKFNrqrNYHKktkZvQQx133XjnaQJ4xQubsaAlDJ1RsV5vL0kLnKMYYN0oz2oRLzvwIAPoPs= ceph-d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:51:55.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:51:55.814 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.081+0000 7f1641800700 1 -- 172.21.15.12:0/2963862111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 msgr2=0x7f163c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.081+0000 7f1641800700 1 --2- 172.21.15.12:0/2963862111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c1052a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f162c009910 tx=0x7f162c009c20 comp rx=0 tx=0).stop 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.082+0000 7f1641800700 1 -- 172.21.15.12:0/2963862111 shutdown_connections 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.082+0000 7f1641800700 1 --2- 172.21.15.12:0/2963862111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c1052a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.082+0000 7f1641800700 1 -- 172.21.15.12:0/2963862111 >> 172.21.15.12:0/2963862111 conn(0x7f163c100470 msgr2=0x7f163c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.082+0000 7f1641800700 1 -- 172.21.15.12:0/2963862111 shutdown_connections 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.082+0000 7f1641800700 1 -- 172.21.15.12:0/2963862111 wait complete. 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.083+0000 7f1641800700 1 Processor -- start 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.083+0000 7f1641800700 1 -- start start 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.083+0000 7f1641800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.083+0000 7f1641800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10ad80 con 0x7f163c104e80 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.084+0000 7f163be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.084+0000 7f163be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54212/0 (socket says 172.21.15.12:54212) 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.084+0000 7f163be00700 1 -- 172.21.15.12:0/2221720182 learned_addr learned my addr 172.21.15.12:0/2221720182 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.084+0000 7f163be00700 1 -- 172.21.15.12:0/2221720182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c19aa50 con 0x7f163c104e80 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.084+0000 7f163be00700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162c005950 tx=0x7f162c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.085+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c026070 con 0x7f163c104e80 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.085+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c00bc00 con 0x7f163c104e80 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.085+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f162c003a10 con 0x7f163c104e80 2024-04-26T21:51:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.085+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c19ace0 con 0x7f163c104e80 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.085+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c19d9d0 con 0x7f163c104e80 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.086+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f162c029070 con 0x7f163c104e80 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.086+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c10b500 con 0x7f163c104e80 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.087+0000 7f1638c00700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 0x7f162003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.087+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f162c01e070 con 0x7f163c104e80 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.087+0000 7f163b400700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 0x7f162003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.088+0000 7f163b400700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 0x7f162003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1628006fd0 tx=0x7f1628006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.093+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f162c013070 con 0x7f163c104e80 2024-04-26T21:51:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.332+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f163c02e5a0 con 0x7f163c104e80 2024-04-26T21:51:58.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.335+0000 7f1638c00700 1 -- 172.21.15.12:0/2221720182 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f162c0300c0 con 0x7f163c104e80 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.338+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 msgr2=0x7f162003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.338+0000 7f1641800700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 0x7f162003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1628006fd0 tx=0x7f1628006e40 comp rx=0 tx=0).stop 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.338+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 msgr2=0x7f163c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.338+0000 7f1641800700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162c005950 tx=0x7f162c004fa0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 shutdown_connections 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f16200396b0 0x7f162003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 --2- 172.21.15.12:0/2221720182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f163c104e80 0x7f163c19a510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 >> 172.21.15.12:0/2221720182 conn(0x7f163c100470 msgr2=0x7f163c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 shutdown_connections 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:51:58.339+0000 7f1641800700 1 -- 172.21.15.12:0/2221720182 wait complete. 2024-04-26T21:51:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2221720182' entity='client.admin' 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:59.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:51:59 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:51:59.641 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:51:59.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:52:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.466+0000 7fab75200700 1 -- 172.21.15.12:0/2889818094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 msgr2=0x7fab70105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.466+0000 7fab75200700 1 --2- 172.21.15.12:0/2889818094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab70105b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab60009910 tx=0x7fab60009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.466+0000 7fab75200700 1 -- 172.21.15.12:0/2889818094 shutdown_connections 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.466+0000 7fab75200700 1 --2- 172.21.15.12:0/2889818094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab70105b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.466+0000 7fab75200700 1 -- 172.21.15.12:0/2889818094 >> 172.21.15.12:0/2889818094 conn(0x7fab700fd180 msgr2=0x7fab700ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:02.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.467+0000 7fab75200700 1 -- 172.21.15.12:0/2889818094 shutdown_connections 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.467+0000 7fab75200700 1 -- 172.21.15.12:0/2889818094 wait complete. 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.469+0000 7fab75200700 1 Processor -- start 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.469+0000 7fab75200700 1 -- start start 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab75200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab75200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab70108bc0 con 0x7fab70103770 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab6f400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab6f400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47450/0 (socket says 172.21.15.12:47450) 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab6f400700 1 -- 172.21.15.12:0/3026220928 learned_addr learned my addr 172.21.15.12:0/3026220928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.470+0000 7fab6f400700 1 -- 172.21.15.12:0/3026220928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7019aa90 con 0x7fab70103770 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab6f400700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab60005b40 tx=0x7fab60004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab60027070 con 0x7fab70103770 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6000be10 con 0x7fab70103770 2024-04-26T21:52:02.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7019ad80 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab60022d80 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.471+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7019da10 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.472+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab70109340 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.472+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fab6002a070 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.473+0000 7fab67e00700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 0x7fab5403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.473+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fab6001e030 con 0x7fab70103770 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.473+0000 7fab6ea00700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 0x7fab5403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.474+0000 7fab6ea00700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 0x7fab5403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006fd0 tx=0x7fab5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:02.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.478+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab60013070 con 0x7fab70103770 2024-04-26T21:52:02.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.715+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fab7002e6e0 con 0x7fab54039650 2024-04-26T21:52:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.721+0000 7fab67e00700 1 -- 172.21.15.12:0/3026220928 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab7002e6e0 con 0x7fab54039650 2024-04-26T21:52:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 msgr2=0x7fab5403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 0x7fab5403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006fd0 tx=0x7fab5c006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 msgr2=0x7fab7019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab60005b40 tx=0x7fab60004f80 comp rx=0 tx=0).stop 2024-04-26T21:52:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 shutdown_connections 2024-04-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fab54039650 0x7fab5403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 --2- 172.21.15.12:0/3026220928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab70103770 0x7fab7019a550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 >> 172.21.15.12:0/3026220928 conn(0x7fab700fd180 msgr2=0x7fab700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 shutdown_connections 2024-04-26T21:52:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:02.724+0000 7fab75200700 1 -- 172.21.15.12:0/3026220928 wait complete. 2024-04-26T21:52:03.608 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-04-26T21:52:03.608 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:52:03.608 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:52:03.627 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:52:03.627 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:03.684 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-04-26T21:52:03.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch host add smithi077 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:03 smithi012 ceph-mon[28184]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T21:52:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:04.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:04 smithi012 ceph-mon[28184]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:04.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:04 smithi012 ceph-mon[28184]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:06.002 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:05 smithi012 ceph-mon[28184]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:06.003 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:05 smithi012 ceph-mon[28184]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:52:06.003 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:06.003 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:06.003 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.175+0000 7f7c4ae00700 1 -- 172.21.15.12:0/1896335643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 msgr2=0x7f7c441054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.175+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/1896335643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441054d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b0a0 tx=0x7f7c3800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.175+0000 7f7c4ae00700 1 -- 172.21.15.12:0/1896335643 shutdown_connections 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.175+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/1896335643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441054d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.176+0000 7f7c4ae00700 1 -- 172.21.15.12:0/1896335643 >> 172.21.15.12:0/1896335643 conn(0x7f7c441009a0 msgr2=0x7f7c44102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:06.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.176+0000 7f7c4ae00700 1 -- 172.21.15.12:0/1896335643 shutdown_connections 2024-04-26T21:52:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.176+0000 7f7c4ae00700 1 -- 172.21.15.12:0/1896335643 wait complete. 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.177+0000 7f7c4ae00700 1 Processor -- start 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.177+0000 7f7c4ae00700 1 -- start start 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.177+0000 7f7c4ae00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.177+0000 7f7c4ae00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410f370 con 0x7f7c441050f0 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.178+0000 7f7c49a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.178+0000 7f7c49a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47464/0 (socket says 172.21.15.12:47464) 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.178+0000 7f7c49a00700 1 -- 172.21.15.12:0/16599757 learned_addr learned my addr 172.21.15.12:0/16599757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.178+0000 7f7c49a00700 1 -- 172.21.15.12:0/16599757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c441a0aa0 con 0x7f7c441050f0 2024-04-26T21:52:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.178+0000 7f7c49a00700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800d010 tx=0x7f7c38003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.179+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c38021020 con 0x7f7c441050f0 2024-04-26T21:52:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.179+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c380044b0 con 0x7f7c441050f0 2024-04-26T21:52:06.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.179+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c38017040 con 0x7f7c441050f0 2024-04-26T21:52:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.179+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441a0d30 con 0x7f7c441050f0 2024-04-26T21:52:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.179+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4419a630 con 0x7f7c441050f0 2024-04-26T21:52:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.180+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c441065d0 con 0x7f7c441050f0 2024-04-26T21:52:06.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.180+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7c38024020 con 0x7f7c441050f0 2024-04-26T21:52:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.181+0000 7f7c42000700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 0x7f7c2c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.181+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c3800f050 con 0x7f7c441050f0 2024-04-26T21:52:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.181+0000 7f7c49000700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 0x7f7c2c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:06.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.182+0000 7f7c49000700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 0x7f7c2c03bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c30006fd0 tx=0x7f7c30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:06.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.186+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c3802b020 con 0x7f7c441050f0 2024-04-26T21:52:06.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:06.421+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}) -- 0x7f7c44064ae0 con 0x7f7c2c039700 2024-04-26T21:52:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:06 smithi012 ceph-mon[28184]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:08.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:07 smithi012 ceph-mon[28184]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:08 smithi012 ceph-mon[28184]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:08 smithi012 ceph-mon[28184]: Deploying cephadm binary to smithi077 2024-04-26T21:52:11.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:10 smithi012 ceph-mon[28184]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:11.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.330+0000 7f7c42000700 1 -- 172.21.15.12:0/16599757 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f7c44064ae0 con 0x7f7c2c039700 2024-04-26T21:52:11.333 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-04-26T21:52:11.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 msgr2=0x7f7c2c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 0x7f7c2c03bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c30006fd0 tx=0x7f7c30006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 msgr2=0x7f7c441a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800d010 tx=0x7f7c38003ce0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 shutdown_connections 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7c2c039700 0x7f7c2c03bbc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 --2- 172.21.15.12:0/16599757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c441050f0 0x7f7c441a0560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.334+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 >> 172.21.15.12:0/16599757 conn(0x7f7c441009a0 msgr2=0x7f7c44106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:11.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.335+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 shutdown_connections 2024-04-26T21:52:11.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:11.335+0000 7f7c4ae00700 1 -- 172.21.15.12:0/16599757 wait complete. 2024-04-26T21:52:12.097 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:52:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:12.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:12 smithi012 ceph-mon[28184]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:12.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:12.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:12 smithi012 ceph-mon[28184]: Added host smithi077 2024-04-26T21:52:12.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:12.695 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:13.711 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:13 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 -- 172.21.15.12:0/920940412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 msgr2=0x7f020c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 --2- 172.21.15.12:0/920940412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1032b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0200009910 tx=0x7f0200009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 -- 172.21.15.12:0/920940412 shutdown_connections 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 --2- 172.21.15.12:0/920940412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1032b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 -- 172.21.15.12:0/920940412 >> 172.21.15.12:0/920940412 conn(0x7f020c0fe780 msgr2=0x7f020c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.580+0000 7f0213000700 1 -- 172.21.15.12:0/920940412 shutdown_connections 2024-04-26T21:52:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.581+0000 7f0213000700 1 -- 172.21.15.12:0/920940412 wait complete. 2024-04-26T21:52:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.581+0000 7f0213000700 1 Processor -- start 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0213000700 1 -- start start 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0213000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0213000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10d150 con 0x7f020c102ed0 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0211c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0211c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50148/0 (socket says 172.21.15.12:50148) 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.582+0000 7f0211c00700 1 -- 172.21.15.12:0/377169916 learned_addr learned my addr 172.21.15.12:0/377169916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.583+0000 7f0211c00700 1 -- 172.21.15.12:0/377169916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c1a0b70 con 0x7f020c102ed0 2024-04-26T21:52:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.583+0000 7f0211c00700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0200006010 tx=0x7f0200005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.583+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0200026070 con 0x7f020c102ed0 2024-04-26T21:52:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.583+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020000bba0 con 0x7f020c102ed0 2024-04-26T21:52:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.583+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c1a0e60 con 0x7f020c102ed0 2024-04-26T21:52:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.584+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0200003910 con 0x7f020c102ed0 2024-04-26T21:52:14.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.584+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c19a8d0 con 0x7f020c102ed0 2024-04-26T21:52:14.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.585+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c1043b0 con 0x7f020c102ed0 2024-04-26T21:52:14.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.585+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0200029070 con 0x7f020c102ed0 2024-04-26T21:52:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.586+0000 7f020a000700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 0x7f01f403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.586+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f020001e070 con 0x7f020c102ed0 2024-04-26T21:52:14.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.586+0000 7f0211200700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 0x7f01f403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:14.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.587+0000 7f0211200700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 0x7f01f403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:14.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.592+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0200013070 con 0x7f020c102ed0 2024-04-26T21:52:14.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:14 smithi012 ceph-mon[28184]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:14.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:14 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.830+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f020c19bc30 con 0x7f01f4039650 2024-04-26T21:52:14.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.831+0000 7f020a000700 1 -- 172.21.15.12:0/377169916 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f020c19bc30 con 0x7f01f4039650 2024-04-26T21:52:14.833 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:52:14.833 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.834+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 msgr2=0x7f01f403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.834+0000 7f0213000700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 0x7f01f403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.834+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 msgr2=0x7f020c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.834+0000 7f0213000700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0200006010 tx=0x7f0200005780 comp rx=0 tx=0).stop 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.834+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 shutdown_connections 2024-04-26T21:52:14.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.835+0000 7f0213000700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01f4039650 0x7f01f403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.835+0000 7f0213000700 1 --2- 172.21.15.12:0/377169916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f020c102ed0 0x7f020c1a0630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.835+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 >> 172.21.15.12:0/377169916 conn(0x7f020c0fe780 msgr2=0x7f020c075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.835+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 shutdown_connections 2024-04-26T21:52:14.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:14.835+0000 7f0213000700 1 -- 172.21.15.12:0/377169916 wait complete. 2024-04-26T21:52:15.551 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-04-26T21:52:15.551 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:52:15.551 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:52:15.570 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:52:15.570 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:15.628 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-04-26T21:52:15.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch host add smithi154 2024-04-26T21:52:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:16.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:16 smithi012 ceph-mon[28184]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:16.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:16 smithi012 ceph-mon[28184]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:18.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.203+0000 7fd514600700 1 -- 172.21.15.12:0/1072097328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 msgr2=0x7fd50c105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.203+0000 7fd514600700 1 --2- 172.21.15.12:0/1072097328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c105680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd500009910 tx=0x7fd500009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.203+0000 7fd514600700 1 -- 172.21.15.12:0/1072097328 shutdown_connections 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.203+0000 7fd514600700 1 --2- 172.21.15.12:0/1072097328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c105680 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.203+0000 7fd514600700 1 -- 172.21.15.12:0/1072097328 >> 172.21.15.12:0/1072097328 conn(0x7fd50c100b70 msgr2=0x7fd50c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.204+0000 7fd514600700 1 -- 172.21.15.12:0/1072097328 shutdown_connections 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.204+0000 7fd514600700 1 -- 172.21.15.12:0/1072097328 wait complete. 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd514600700 1 Processor -- start 2024-04-26T21:52:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd514600700 1 -- start start 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd514600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd514600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c10f520 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd511c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd511c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50174/0 (socket says 172.21.15.12:50174) 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.205+0000 7fd511c00700 1 -- 172.21.15.12:0/2940763480 learned_addr learned my addr 172.21.15.12:0/2940763480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.206+0000 7fd511c00700 1 -- 172.21.15.12:0/2940763480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c1a0be0 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.206+0000 7fd511c00700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd500006010 tx=0x7fd500005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.206+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd500026070 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.206+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50000bba0 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.206+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50c1a0ed0 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.207+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd500003910 con 0x7fd50c1052a0 2024-04-26T21:52:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.207+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50c19a860 con 0x7fd50c1052a0 2024-04-26T21:52:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.208+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd500029070 con 0x7fd50c1052a0 2024-04-26T21:52:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.208+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50c106780 con 0x7fd50c1052a0 2024-04-26T21:52:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.208+0000 7fd50a000700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 0x7fd4f403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.208+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd50001e070 con 0x7fd50c1052a0 2024-04-26T21:52:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.209+0000 7fd511200700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 0x7fd4f403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.210+0000 7fd511200700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 0x7fd4f403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006fd0 tx=0x7fd4f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.215+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd500013070 con 0x7fd50c1052a0 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:52:18.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:18.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:18.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T21:52:18.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:18 smithi012 ceph-mon[28184]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:18.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:18.456+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}) -- 0x7fd50c19bd90 con 0x7fd4f4039650 2024-04-26T21:52:19.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:20.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:52:20.245 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:20.245 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:20.245 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:20.245 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:19 smithi012 ceph-mon[28184]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:20.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:20 smithi012 ceph-mon[28184]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:20.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:20 smithi012 ceph-mon[28184]: Deploying cephadm binary to smithi154 2024-04-26T21:52:22.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:22 smithi012 ceph-mon[28184]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:23.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.370+0000 7fd50a000700 1 -- 172.21.15.12:0/2940763480 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd50c19bd90 con 0x7fd4f4039650 2024-04-26T21:52:23.373 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 msgr2=0x7fd4f403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 0x7fd4f403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006fd0 tx=0x7fd4f8006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 msgr2=0x7fd50c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd500006010 tx=0x7fd500005780 comp rx=0 tx=0).stop 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 shutdown_connections 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4f4039650 0x7fd4f403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 --2- 172.21.15.12:0/2940763480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd50c1052a0 0x7fd50c1a06a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.373+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 >> 172.21.15.12:0/2940763480 conn(0x7fd50c100b70 msgr2=0x7fd50c106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.374+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 shutdown_connections 2024-04-26T21:52:23.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:23.374+0000 7fd514600700 1 -- 172.21.15.12:0/2940763480 wait complete. 2024-04-26T21:52:24.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:52:24.598 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:24.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:24 smithi012 ceph-mon[28184]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:24.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:24.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:24 smithi012 ceph-mon[28184]: Added host smithi154 2024-04-26T21:52:24.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:25.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.550+0000 7f8fd9a00700 1 -- 172.21.15.12:0/2482636120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 msgr2=0x7f8fd4105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.550+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/2482636120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd4105690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009910 tx=0x7f8fc4009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.551+0000 7f8fd9a00700 1 -- 172.21.15.12:0/2482636120 shutdown_connections 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.551+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/2482636120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd4105690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.551+0000 7f8fd9a00700 1 -- 172.21.15.12:0/2482636120 >> 172.21.15.12:0/2482636120 conn(0x7f8fd4100b40 msgr2=0x7f8fd4102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.551+0000 7f8fd9a00700 1 -- 172.21.15.12:0/2482636120 shutdown_connections 2024-04-26T21:52:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.551+0000 7f8fd9a00700 1 -- 172.21.15.12:0/2482636120 wait complete. 2024-04-26T21:52:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.552+0000 7f8fd9a00700 1 Processor -- start 2024-04-26T21:52:26.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.552+0000 7f8fd9a00700 1 -- start start 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.553+0000 7f8fd9a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.553+0000 7f8fd9a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410f530 con 0x7f8fd41052b0 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.553+0000 7f8fd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.553+0000 7f8fd2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48694/0 (socket says 172.21.15.12:48694) 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.553+0000 7f8fd2a00700 1 -- 172.21.15.12:0/3891230165 learned_addr learned my addr 172.21.15.12:0/3891230165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fd2a00700 1 -- 172.21.15.12:0/3891230165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd41a0c50 con 0x7f8fd41052b0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fd2a00700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40056d0 tx=0x7f8fc40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4026070 con 0x7f8fd41052b0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc400bba0 con 0x7f8fd41052b0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc4003920 con 0x7f8fd41052b0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.554+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd41a0f40 con 0x7f8fd41052b0 2024-04-26T21:52:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.555+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd419a8d0 con 0x7f8fd41052b0 2024-04-26T21:52:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.556+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd4106790 con 0x7f8fd41052b0 2024-04-26T21:52:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.556+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8fc4029070 con 0x7f8fd41052b0 2024-04-26T21:52:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.556+0000 7f8fcb400700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 0x7f8fb803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.557+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fc401e070 con 0x7f8fd41052b0 2024-04-26T21:52:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.557+0000 7f8fd2000700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 0x7f8fb803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.557+0000 7f8fd2000700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 0x7f8fb803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0006fd0 tx=0x7f8fc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.563+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8fc4013070 con 0x7f8fd41052b0 2024-04-26T21:52:26.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.798+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fd419be70 con 0x7f8fb80396b0 2024-04-26T21:52:26.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:26 smithi012 ceph-mon[28184]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:26.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.802+0000 7f8fcb400700 1 -- 172.21.15.12:0/3891230165 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8fd419be70 con 0x7f8fb80396b0 2024-04-26T21:52:26.803 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:52:26.803 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 msgr2=0x7f8fb803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 0x7f8fb803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0006fd0 tx=0x7f8fc0006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 msgr2=0x7f8fd41a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40056d0 tx=0x7f8fc40057b0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 shutdown_connections 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fb80396b0 0x7f8fb803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 --2- 172.21.15.12:0/3891230165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd41052b0 0x7f8fd41a0710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 >> 172.21.15.12:0/3891230165 conn(0x7f8fd4100b40 msgr2=0x7f8fd4108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.805+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 shutdown_connections 2024-04-26T21:52:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:26.806+0000 7f8fd9a00700 1 -- 172.21.15.12:0/3891230165 wait complete. 2024-04-26T21:52:27.543 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:52:27.544 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:52:27.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:27 smithi012 ceph-mon[28184]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:28.015 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:28 smithi012 ceph-mon[28184]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 -- 172.21.15.12:0/3963736450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 msgr2=0x7fad00103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 --2- 172.21.15.12:0/3963736450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad00103470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facf0009910 tx=0x7facf0009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 -- 172.21.15.12:0/3963736450 shutdown_connections 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 --2- 172.21.15.12:0/3963736450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad00103470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 -- 172.21.15.12:0/3963736450 >> 172.21.15.12:0/3963736450 conn(0x7fad000fe960 msgr2=0x7fad00100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 -- 172.21.15.12:0/3963736450 shutdown_connections 2024-04-26T21:52:30.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.050+0000 7fad06e00700 1 -- 172.21.15.12:0/3963736450 wait complete. 2024-04-26T21:52:30.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.051+0000 7fad06e00700 1 Processor -- start 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.052+0000 7fad06e00700 1 -- start start 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.052+0000 7fad06e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.052+0000 7fad06e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fad0010d310 con 0x7fad00103090 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.053+0000 7facffe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.053+0000 7facffe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57590/0 (socket says 172.21.15.12:57590) 2024-04-26T21:52:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.053+0000 7facffe00700 1 -- 172.21.15.12:0/234526605 learned_addr learned my addr 172.21.15.12:0/234526605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.053+0000 7facffe00700 1 -- 172.21.15.12:0/234526605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad001a0c30 con 0x7fad00103090 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.053+0000 7facffe00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.054+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facf0026070 con 0x7fad00103090 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.054+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf000bba0 con 0x7fad00103090 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.054+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad001a0f20 con 0x7fad00103090 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.054+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facf0003910 con 0x7fad00103090 2024-04-26T21:52:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.054+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0019a8b0 con 0x7fad00103090 2024-04-26T21:52:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.055+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad00104570 con 0x7fad00103090 2024-04-26T21:52:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.055+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7facf0029070 con 0x7fad00103090 2024-04-26T21:52:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.056+0000 7facfcc00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 0x7face403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.056+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7facf001e070 con 0x7fad00103090 2024-04-26T21:52:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.056+0000 7facff400700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 0x7face403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:30.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.057+0000 7facff400700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 0x7face403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.062+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7facf0013070 con 0x7fad00103090 2024-04-26T21:52:30.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:30.303+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fad0019bde0 con 0x7fad00103090 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:30.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:52:30.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:30.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:30.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T21:52:30.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:30 smithi012 ceph-mon[28184]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:31.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.231+0000 7facfcc00700 1 -- 172.21.15.12:0/234526605 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7facf00282c0 con 0x7fad00103090 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.234+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 msgr2=0x7face403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.234+0000 7fad06e00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 0x7face403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 msgr2=0x7fad001a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0005780 comp rx=0 tx=0).stop 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 shutdown_connections 2024-04-26T21:52:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7face40396b0 0x7face403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 --2- 172.21.15.12:0/234526605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad00103090 0x7fad001a06f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 >> 172.21.15.12:0/234526605 conn(0x7fad000fe960 msgr2=0x7fad00075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 shutdown_connections 2024-04-26T21:52:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:31.235+0000 7fad06e00700 1 -- 172.21.15.12:0/234526605 wait complete. 2024-04-26T21:52:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-04-26T21:52:31.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:31.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:52:31.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/234526605' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:52:31.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:31.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:31.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:32.130 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-04-26T21:52:32.130 INFO:tasks.cephadm:Adding mon.b on smithi077 2024-04-26T21:52:32.130 INFO:tasks.cephadm:Adding mon.c on smithi154 2024-04-26T21:52:32.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c' 2024-04-26T21:52:32.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:32 smithi012 ceph-mon[28184]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:32.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:32 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/234526605' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:52:32.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:32 smithi012 ceph-mon[28184]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:32.368 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:33.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.779+0000 7f6d00db8700 1 -- 172.21.15.154:0/2656958249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 msgr2=0x7f6cfc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:33.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.779+0000 7f6d00db8700 1 --2- 172.21.15.154:0/2656958249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc103470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009910 tx=0x7f6cec009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:33.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 -- 172.21.15.154:0/2656958249 shutdown_connections 2024-04-26T21:52:33.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 --2- 172.21.15.154:0/2656958249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc103470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 -- 172.21.15.154:0/2656958249 >> 172.21.15.154:0/2656958249 conn(0x7f6cfc0fe960 msgr2=0x7f6cfc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:33.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 -- 172.21.15.154:0/2656958249 shutdown_connections 2024-04-26T21:52:33.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 -- 172.21.15.154:0/2656958249 wait complete. 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 Processor -- start 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.780+0000 7f6d00db8700 1 -- start start 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.781+0000 7f6d00db8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.781+0000 7f6d00db8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc10d310 con 0x7f6cfc103090 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.781+0000 7f6cfa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.781+0000 7f6cfa59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.154:50736/0 (socket says 172.21.15.154:50736) 2024-04-26T21:52:33.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.781+0000 7f6cfa59c700 1 -- 172.21.15.154:0/988522135 learned_addr learned my addr 172.21.15.154:0/988522135 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:52:33.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.782+0000 7f6cfa59c700 1 -- 172.21.15.154:0/988522135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc19eaf0 con 0x7f6cfc103090 2024-04-26T21:52:33.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.782+0000 7f6cfa59c700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006010 tx=0x7f6cec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:33.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.783+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cec026070 con 0x7f6cfc103090 2024-04-26T21:52:33.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.783+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc19ede0 con 0x7f6cfc103090 2024-04-26T21:52:33.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.783+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc198850 con 0x7f6cfc103090 2024-04-26T21:52:33.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.783+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cec00bba0 con 0x7f6cfc103090 2024-04-26T21:52:33.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.783+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cec030060 con 0x7f6cfc103090 2024-04-26T21:52:33.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.784+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc104570 con 0x7f6cfc103090 2024-04-26T21:52:33.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.784+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6cec029070 con 0x7f6cfc103090 2024-04-26T21:52:33.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.784+0000 7f6ceb7fe700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 0x7f6ce403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.784+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6cec01e070 con 0x7f6cfc103090 2024-04-26T21:52:33.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.785+0000 7f6cf9d9b700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 0x7f6ce403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.786+0000 7f6cf9d9b700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 0x7f6ce403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0006fd0 tx=0x7f6cf0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:33.803+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6cec013070 con 0x7f6cfc103090 2024-04-26T21:52:34.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.012+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}) -- 0x7f6cfc0008d0 con 0x7f6ce4039650 2024-04-26T21:52:34.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.028+0000 7f6ceb7fe700 1 -- 172.21.15.154:0/988522135 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6cfc0008d0 con 0x7f6ce4039650 2024-04-26T21:52:34.028 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mon update... 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 msgr2=0x7f6ce403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 0x7f6ce403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0006fd0 tx=0x7f6cf0006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 msgr2=0x7f6cfc19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006010 tx=0x7f6cec005780 comp rx=0 tx=0).stop 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 shutdown_connections 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6ce4039650 0x7f6ce403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 --2- 172.21.15.154:0/988522135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cfc103090 0x7f6cfc19e5b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 >> 172.21.15.154:0/988522135 conn(0x7f6cfc0fe960 msgr2=0x7f6cfc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:34.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 shutdown_connections 2024-04-26T21:52:34.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:34.029+0000 7f6d00db8700 1 -- 172.21.15.154:0/988522135 wait complete. 2024-04-26T21:52:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:34.648 DEBUG:teuthology.orchestra.run.smithi077:mon.b> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:52:34.651 DEBUG:teuthology.orchestra.run.smithi154:mon.c> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:52:34.653 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:52:34.654 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:52:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:35 smithi012 ceph-mon[28184]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:35 smithi012 ceph-mon[28184]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c;count:3 2024-04-26T21:52:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:35 smithi012 ceph-mon[28184]: Deploying daemon mon.c on smithi154 2024-04-26T21:52:36.248 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:36.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:36 smithi012 ceph-mon[28184]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 -- 172.21.15.154:0/401467810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 msgr2=0x7f8f40104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 --2- 172.21.15.154:0/401467810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f40104430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009910 tx=0x7f8f3c009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 -- 172.21.15.154:0/401467810 shutdown_connections 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 --2- 172.21.15.154:0/401467810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f40104430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 -- 172.21.15.154:0/401467810 >> 172.21.15.154:0/401467810 conn(0x7f8f400fd820 msgr2=0x7f8f400ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 -- 172.21.15.154:0/401467810 shutdown_connections 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.080+0000 7f8f481e9700 1 -- 172.21.15.154:0/401467810 wait complete. 2024-04-26T21:52:38.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.081+0000 7f8f481e9700 1 Processor -- start 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.081+0000 7f8f481e9700 1 -- start start 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.081+0000 7f8f481e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.081+0000 7f8f481e9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010d380 con 0x7f8f40104050 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.082+0000 7f8f45f85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.082+0000 7f8f45f85700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.154:50744/0 (socket says 172.21.15.154:50744) 2024-04-26T21:52:38.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.082+0000 7f8f45f85700 1 -- 172.21.15.154:0/2990054123 learned_addr learned my addr 172.21.15.154:0/2990054123 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.082+0000 7f8f45f85700 1 -- 172.21.15.154:0/2990054123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4019eb90 con 0x7f8f40104050 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f45f85700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0056d0 tx=0x7f8f3c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c026070 con 0x7f8f40104050 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4019ee20 con 0x7f8f40104050 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f40198720 con 0x7f8f40104050 2024-04-26T21:52:38.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f3c004370 con 0x7f8f40104050 2024-04-26T21:52:38.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.083+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f3c003990 con 0x7f8f40104050 2024-04-26T21:52:38.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.084+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4006bb10 con 0x7f8f40104050 2024-04-26T21:52:38.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.085+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8f3c029070 con 0x7f8f40104050 2024-04-26T21:52:38.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.085+0000 7f8f36ffd700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 0x7f8f2c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.085+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f3c01e070 con 0x7f8f40104050 2024-04-26T21:52:38.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.085+0000 7f8f45784700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 0x7f8f2c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.086+0000 7f8f45784700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 0x7f8f2c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.100+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f3c013070 con 0x7f8f40104050 2024-04-26T21:52:38.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.436+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f40064a80 con 0x7f8f40104050 2024-04-26T21:52:38.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.437+0000 7f8f36ffd700 1 -- 172.21.15.154:0/2990054123 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f8f3c00d340 con 0x7f8f40104050 2024-04-26T21:52:38.438 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:52:38.438 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":1,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","modified":"2024-04-26T21:50:22.222336Z","created":"2024-04-26T21:50:22.222336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 msgr2=0x7f8f2c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 0x7f8f2c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30006e40 comp rx=0 tx=0).stop 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 msgr2=0x7f8f4019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0056d0 tx=0x7f8f3c0057b0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 shutdown_connections 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8f2c039650 0x7f8f2c03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 --2- 172.21.15.154:0/2990054123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f40104050 0x7f8f4019e650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 >> 172.21.15.154:0/2990054123 conn(0x7f8f400fd820 msgr2=0x7f8f40104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 shutdown_connections 2024-04-26T21:52:38.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:38.439+0000 7f8f481e9700 1 -- 172.21.15.154:0/2990054123 wait complete. 2024-04-26T21:52:38.441 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 1 2024-04-26T21:52:38.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:38 smithi012 ceph-mon[28184]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:38.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:38 smithi154 systemd[1]: Starting Ceph mon.c for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:52:39.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 podman[32559]: 2024-04-26 21:52:39.258433401 +0000 UTC m=+0.111671535 container create e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T21:52:39.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 podman[32559]: 2024-04-26 21:52:39.164616731 +0000 UTC m=+0.017854806 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 podman[32559]: 2024-04-26 21:52:39.435781178 +0000 UTC m=+0.289019256 container init e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 podman[32559]: 2024-04-26 21:52:39.438786772 +0000 UTC m=+0.292024844 container start e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pidfile_write: ignore empty --pid-file 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: load: jerasure load: lrc 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Git sha 0 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: DB SUMMARY 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: DB Session ID: DQ15BKXR53PAYBXLEGHF 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:52:39.520 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:52:39.521 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.env: 0x55f8ee33ecc0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.info_log: 0x55f8ef18bd80 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.statistics: (nil) 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.use_fsync: 0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:52:39.522 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.db_log_dir: 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.wal_dir: 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.write_buffer_manager: 0x55f8ef15d0e0 2024-04-26T21:52:39.523 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.unordered_write: 0 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.row_cache: None 2024-04-26T21:52:39.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.wal_filter: None 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.wal_compression: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:52:39.525 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:52:39.526 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_open_files: -1 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Compression algorithms supported: 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:52:39.528 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kZSTD supported: 0 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.merge_operator: 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_filter: None 2024-04-26T21:52:39.529 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f8ef18bf40) 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_type: 0 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: data_block_index_type: 0 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_shortening: 1 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: checksum: 4 2024-04-26T21:52:39.530 INFO:journalctl@ceph.mon.c.smithi154.stdout: no_block_cache: 0 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache: 0x55f8ef18f1f0 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_options: 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: capacity : 536870912 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: num_shard_bits : 4 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: strict_capacity_limit : 0 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_cache_compressed: (nil) 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: persistent_cache: (nil) 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_size: 4096 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_size_deviation: 10 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_restart_interval: 16 2024-04-26T21:52:39.531 INFO:journalctl@ceph.mon.c.smithi154.stdout: index_block_restart_interval: 1 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: metadata_block_size: 4096 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: partition_filters: 0 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: use_delta_encoding: 1 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: filter_policy: bloomfilter 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: whole_key_filtering: 1 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: verify_compression: 0 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: format_version: 5 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: enable_index_compression: 1 2024-04-26T21:52:39.532 INFO:journalctl@ceph.mon.c.smithi154.stdout: block_align: 0 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout: max_auto_readahead_size: 262144 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout: prepopulate_block_cache: 0 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression: NoCompression 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.num_levels: 7 2024-04-26T21:52:39.533 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:52:39.534 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:52:39.535 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:52:39.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:52:39.537 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:52:39.538 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.ttl: 2592000 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:52:39.539 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a987d3b7-890a-47c6-94c4-444bc7ea7b89 2024-04-26T21:52:39.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168359470358, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168359470892, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714168359, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a987d3b7-890a-47c6-94c4-444bc7ea7b89", "db_session_id": "DQ15BKXR53PAYBXLEGHF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714168359470991, "job": 1, "event": "recovery_finished"} 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f8ef2ac000 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: DB pointer 0x55f8ef298000 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** DB Stats ** 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:52:39.541 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** Compaction Stats [default] ** 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:52:39.542 INFO:journalctl@ceph.mon.c.smithi154.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** Compaction Stats [default] ** 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:52:39.543 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: Block cache BinnedLRUCache@0x55f8ef18f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: using public_addr v2:172.21.15.154:0/0 -> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: starting mon.c rank -1 at public addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] at bind addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(???) e0 preinit fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:39.544 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: e1 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: legacy client fscid: -1 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout: No filesystems configured 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:52:39.545 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Added host smithi077 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.546 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:52:39.547 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Deploying cephadm binary to smithi154 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Added host smithi154 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.548 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.549 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:39.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/234526605' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.551 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/234526605' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:39.552 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c;count:3 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: Deploying daemon mon.c on smithi154 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/2990054123' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 ceph-mon[32578]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:52:39.553 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 bash[32559]: e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed 2024-04-26T21:52:39.554 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:39 smithi154 systemd[1]: Started Ceph mon.c for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:52:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:39 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/2990054123' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:40.024 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:52:40.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:52:40.257 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.739+0000 7fa0e3c57700 1 -- 172.21.15.154:0/4271557990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0c80035c0 msgr2=0x7fa0c8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.739+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/4271557990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0c80035c0 0x7fa0c8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc101d40 tx=0x7fa0cc00feb0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.739+0000 7fa0e3c57700 1 -- 172.21.15.154:0/4271557990 shutdown_connections 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.739+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/4271557990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0c80035c0 0x7fa0c8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.739+0000 7fa0e3c57700 1 -- 172.21.15.154:0/4271557990 >> 172.21.15.154:0/4271557990 conn(0x7fa0dc0fd900 msgr2=0x7fa0dc0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.740+0000 7fa0e3c57700 1 -- 172.21.15.154:0/4271557990 shutdown_connections 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.740+0000 7fa0e3c57700 1 -- 172.21.15.154:0/4271557990 wait complete. 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.740+0000 7fa0e3c57700 1 Processor -- start 2024-04-26T21:52:44.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e3c57700 1 -- start start 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e3c57700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e3c57700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 0x7fa0dc18fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e3c57700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc109d60 con 0x7fa0dc18fa10 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e3c57700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc109be0 con 0x7fa0c80035c0 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57958/0 (socket says 172.21.15.154:57958) 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 -- 172.21.15.154:0/3779530394 learned_addr learned my addr 172.21.15.154:0/3779530394 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 -- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 msgr2=0x7fa0dc18f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 -- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 msgr2=0x7fa0dc18f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e19f3700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:52:44.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.741+0000 7fa0e11f2700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 0x7fa0dc18fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.742+0000 7fa0e11f2700 1 -- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 msgr2=0x7fa0dc18f4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.742+0000 7fa0e11f2700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.742+0000 7fa0e11f2700 1 -- 172.21.15.154:0/3779530394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1913a0 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0e11f2700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 0x7fa0dc18fe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800b4d0 tx=0x7fa0d800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa0d8011630 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1a4c90 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d8011c80 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1a51f0 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.743+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa0d8010d40 con 0x7fa0dc18fa10 2024-04-26T21:52:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.744+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc103550 con 0x7fa0dc18fa10 2024-04-26T21:52:44.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.745+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa0d8018020 con 0x7fa0dc18fa10 2024-04-26T21:52:44.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.745+0000 7fa0d2ffd700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 0x7fa0c003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.745+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa0d804c490 con 0x7fa0dc18fa10 2024-04-26T21:52:44.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.746+0000 7fa0e19f3700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 0x7fa0c003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.747+0000 7fa0e19f3700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 0x7fa0c003b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00fa60 tx=0x7fa0cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:44.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:44.760+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa0d801f020 con 0x7fa0dc18fa10 2024-04-26T21:52:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: monmap epoch 2 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: last_changed 2024-04-26T21:52:39.510917+0000 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: election_strategy: 1 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: fsmap 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: mgrmap e13: x(active, since 61s) 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:44.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: mon.a calling monitor election 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: mon.c calling monitor election 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: monmap epoch 2 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: last_changed 2024-04-26T21:52:39.510917+0000 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: min_mon_release 19 (squid) 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: election_strategy: 1 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:44.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: fsmap 2024-04-26T21:52:44.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:44.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: mgrmap e13: x(active, since 61s) 2024-04-26T21:52:44.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: overall HEALTH_OK 2024-04-26T21:52:44.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:44.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:45.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.044+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa0dc1a54d0 con 0x7fa0dc18fa10 2024-04-26T21:52:45.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.045+0000 7fa0d2ffd700 1 -- 172.21.15.154:0/3779530394 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa0d8010410 con 0x7fa0dc18fa10 2024-04-26T21:52:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:52:45.044 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","modified":"2024-04-26T21:52:39.510917Z","created":"2024-04-26T21:50:22.222336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 msgr2=0x7fa0c003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 0x7fa0c003b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00fa60 tx=0x7fa0cc005960 comp rx=0 tx=0).stop 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 msgr2=0x7fa0dc18fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 0x7fa0dc18fe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800b4d0 tx=0x7fa0d800b890 comp rx=0 tx=0).stop 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 shutdown_connections 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0c00393f0 0x7fa0c003b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c80035c0 0x7fa0dc18f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 --2- 172.21.15.154:0/3779530394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc18fa10 0x7fa0dc18fe90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 >> 172.21.15.154:0/3779530394 conn(0x7fa0dc0fd900 msgr2=0x7fa0dc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 shutdown_connections 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:45.047+0000 7fa0e3c57700 1 -- 172.21.15.154:0/3779530394 wait complete. 2024-04-26T21:52:45.047 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 2 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: Deploying daemon mon.b on smithi077 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/3779530394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: Deploying daemon mon.b on smithi077 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/3779530394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:46.659 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:52:46.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:52:46.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:46 smithi154 ceph-mon[32578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:46.903 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:52:47.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:46 smithi012 ceph-mon[28184]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:48.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 -- 172.21.15.154:0/2898736178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 msgr2=0x7fac84103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/2898736178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac84103580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fac78009910 tx=0x7fac78009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 -- 172.21.15.154:0/2898736178 shutdown_connections 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/2898736178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac84103580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 -- 172.21.15.154:0/2898736178 >> 172.21.15.154:0/2898736178 conn(0x7fac840fea30 msgr2=0x7fac84100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 -- 172.21.15.154:0/2898736178 shutdown_connections 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.402+0000 7fac8ab6d700 1 -- 172.21.15.154:0/2898736178 wait complete. 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 Processor -- start 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 -- start start 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac8419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410d420 con 0x7fac841031a0 2024-04-26T21:52:48.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac8ab6d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410d2a0 con 0x7fac8419eca0 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57962/0 (socket says 172.21.15.154:57962) 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 -- 172.21.15.154:0/1388142090 learned_addr learned my addr 172.21.15.154:0/1388142090 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 -- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 msgr2=0x7fac84198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 -- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 msgr2=0x7fac84198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.403+0000 7fac83fff700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.404+0000 7fac88909700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac8419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.404+0000 7fac88909700 1 -- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 msgr2=0x7fac84198920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.404+0000 7fac88909700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.404+0000 7fac88909700 1 -- 172.21.15.154:0/1388142090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8419f2e0 con 0x7fac841031a0 2024-04-26T21:52:48.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.404+0000 7fac88909700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac8419e760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac78014000 tx=0x7fac78005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.405+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fac78005dc0 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.405+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac84198e60 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.405+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841993f0 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.405+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7800b6f0 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.405+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fac78003910 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.406+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac68005330 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.407+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fac78029070 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.407+0000 7fac81ffb700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 0x7fac7403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.407+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fac7801e070 con 0x7fac841031a0 2024-04-26T21:52:48.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.407+0000 7fac83fff700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 0x7fac7403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.409+0000 7fac83fff700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 0x7fac7403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac70009180 tx=0x7fac70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.424+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac78013070 con 0x7fac841031a0 2024-04-26T21:52:48.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.575+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800bba0 con 0x7fac841031a0 2024-04-26T21:52:48.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:48.702+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fac680058f0 con 0x7fac841031a0 2024-04-26T21:52:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:48 smithi077 systemd[1]: Started Ceph mon.b for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:52:53.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.595+0000 7fac81ffb700 1 -- 172.21.15.154:0/1388142090 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fac78038510 con 0x7fac841031a0 2024-04-26T21:52:53.596 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-26T21:52:53.596 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","modified":"2024-04-26T21:52:48.573863Z","created":"2024-04-26T21:50:22.222336Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 msgr2=0x7fac7403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 0x7fac7403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac70009180 tx=0x7fac70008040 comp rx=0 tx=0).stop 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 msgr2=0x7fac8419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac8419e760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac78014000 tx=0x7fac78005030 comp rx=0 tx=0).stop 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 shutdown_connections 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac841031a0 0x7fac8419e760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fac740393f0 0x7fac7403b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.600 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 --2- 172.21.15.154:0/1388142090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8419eca0 0x7fac84198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.600 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 >> 172.21.15.154:0/1388142090 conn(0x7fac840fea30 msgr2=0x7fac84104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:53.600 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 shutdown_connections 2024-04-26T21:52:53.600 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:53.598+0000 7fac8ab6d700 1 -- 172.21.15.154:0/1388142090 wait complete. 2024-04-26T21:52:53.600 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 3 2024-04-26T21:52:53.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T21:52:53.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:52:53.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T21:52:53.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/1388142090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: monmap epoch 3 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:53.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: election_strategy: 1 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: fsmap 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: mgrmap e13: x(active, since 70s) 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:53.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: mon.c calling monitor election 2024-04-26T21:52:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: mon.a calling monitor election 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/1388142090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:52:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: monmap epoch 3 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: min_mon_release 19 (squid) 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: election_strategy: 1 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: fsmap 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: mgrmap e13: x(active, since 70s) 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: overall HEALTH_OK 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.281 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:52:54.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:52:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:54.933 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:54 smithi154 ceph-mon[32578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:54.934 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.934 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.934 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:54.934 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:54.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:54 smithi012 ceph-mon[28184]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:54.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:54.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:54.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: mon.c calling monitor election 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: mon.a calling monitor election 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/1388142090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: monmap epoch 3 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: min_mon_release 19 (squid) 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: election_strategy: 1 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:55.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: fsmap 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: mgrmap e13: x(active, since 70s) 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: overall HEALTH_OK 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:55.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:56.515 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mon.b calling monitor election 2024-04-26T21:52:56.515 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:56.515 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mon.b calling monitor election 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mon.c calling monitor election 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mon.a calling monitor election 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: monmap epoch 3 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: min_mon_release 19 (squid) 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: election_strategy: 1 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:56.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: fsmap 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: mgrmap e13: x(active, since 72s) 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: overall HEALTH_OK 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mon.b calling monitor election 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T21:52:56.613 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mon.b calling monitor election 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: monmap epoch 3 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: election_strategy: 1 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: fsmap 2024-04-26T21:52:56.614 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: mgrmap e13: x(active, since 72s) 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.615 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 -- 172.21.15.12:0/3043073243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc418075850 msgr2=0x7fc418075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 --2- 172.21.15.12:0/3043073243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc418075850 0x7fc418075c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009910 tx=0x7fc40c009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 -- 172.21.15.12:0/3043073243 shutdown_connections 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 --2- 172.21.15.12:0/3043073243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc418075850 0x7fc418075c30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 -- 172.21.15.12:0/3043073243 >> 172.21.15.12:0/3043073243 conn(0x7fc4180fe960 msgr2=0x7fc418100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 -- 172.21.15.12:0/3043073243 shutdown_connections 2024-04-26T21:52:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.677+0000 7fc420200700 1 -- 172.21.15.12:0/3043073243 wait complete. 2024-04-26T21:52:56.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.678+0000 7fc420200700 1 Processor -- start 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.678+0000 7fc420200700 1 -- start start 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 0x7fc41819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 0x7fc41819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810f590 con 0x7fc41819fb70 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810f410 con 0x7fc418075850 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc420200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810f710 con 0x7fc41819b2a0 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc41e200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc41e200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35720/0 (socket says 172.21.15.12:35720) 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc41e200700 1 -- 172.21.15.12:0/2090110785 learned_addr learned my addr 172.21.15.12:0/2090110785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc41ce00700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 0x7fc41819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.679+0000 7fc41d800700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 0x7fc41819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 -- 172.21.15.12:0/2090110785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 msgr2=0x7fc41819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 0x7fc41819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 -- 172.21.15.12:0/2090110785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 msgr2=0x7fc41819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 0x7fc41819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 -- 172.21.15.12:0/2090110785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4181a78e0 con 0x7fc41819fb70 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41ce00700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 0x7fc41819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41d800700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 0x7fc41819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.680+0000 7fc41e200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc41000ba50 tx=0x7fc41000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.681+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410019070 con 0x7fc41819fb70 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.681+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41000cde0 con 0x7fc41819fb70 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.681+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410007350 con 0x7fc41819fb70 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.681+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4181a7b10 con 0x7fc41819fb70 2024-04-26T21:52:56.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.681+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a8000 con 0x7fc41819fb70 2024-04-26T21:52:56.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.682+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc41001c070 con 0x7fc41819fb70 2024-04-26T21:52:56.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.682+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc418064a80 con 0x7fc41819fb70 2024-04-26T21:52:56.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.682+0000 7fc416000700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 0x7fc40003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.682+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc41004be10 con 0x7fc41819fb70 2024-04-26T21:52:56.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.683+0000 7fc41d800700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 0x7fc40003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.683+0000 7fc41d800700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 0x7fc40003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006010 tx=0x7fc40c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.689+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc4100074f0 con 0x7fc41819fb70 2024-04-26T21:52:56.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mon.b calling monitor election 2024-04-26T21:52:56.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:56.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T21:52:56.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T21:52:56.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mon.b calling monitor election 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mon.c calling monitor election 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mon.a calling monitor election 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: monmap epoch 3 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: last_changed 2024-04-26T21:52:48.573863+0000 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: created 2024-04-26T21:50:22.222336+0000 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: min_mon_release 19 (squid) 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: election_strategy: 1 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-26T21:52:56.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: fsmap 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: mgrmap e13: x(active, since 72s) 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: overall HEALTH_OK 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:56.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.929+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc4181a82f0 con 0x7fc41819fb70 2024-04-26T21:52:56.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.930+0000 7fc416000700 1 -- 172.21.15.12:0/2090110785 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fc41000c930 con 0x7fc41819fb70 2024-04-26T21:52:56.930 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:56.930 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-04-26T21:52:56.930 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T21:52:56.930 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-26T21:52:56.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 msgr2=0x7fc40003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 0x7fc40003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006010 tx=0x7fc40c01a040 comp rx=0 tx=0).stop 2024-04-26T21:52:56.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 msgr2=0x7fc4181a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc41000ba50 tx=0x7fc41000bd60 comp rx=0 tx=0).stop 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 shutdown_connections 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc41819fb70 0x7fc4181a0020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc418075850 0x7fc41819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc400039ae0 0x7fc40003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 --2- 172.21.15.12:0/2090110785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc41819b2a0 0x7fc41819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 >> 172.21.15.12:0/2090110785 conn(0x7fc4180fe960 msgr2=0x7fc4181090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 shutdown_connections 2024-04-26T21:52:56.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:52:56.933+0000 7fc420200700 1 -- 172.21.15.12:0/2090110785 wait complete. 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2090110785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.286 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:52:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2090110785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:57.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:52:57.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.652 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:52:57.652 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:52:57.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:52:57.719 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:52:57.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:52:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:52:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2090110785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:52:57.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:57.822 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:52:57.822 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:52:57.854 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:52:57.854 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:57.923 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:52:57.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:52:57.954 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:52:57.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:52:58.023 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-04-26T21:52:58.023 INFO:tasks.cephadm:Adding mgr.y on smithi077 2024-04-26T21:52:58.023 INFO:tasks.cephadm:Adding mgr.z on smithi154 2024-04-26T21:52:58.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi077=y;smithi154=z' 2024-04-26T21:52:58.297 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:52:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:58 smithi012 ceph-mon[28184]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:58 smithi012 ceph-mon[28184]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:52:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:58 smithi012 ceph-mon[28184]: Reconfiguring daemon mon.a on smithi012 2024-04-26T21:52:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:52:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:58.625 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:58 smithi154 ceph-mon[32578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:58.626 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:58 smithi154 ceph-mon[32578]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:52:58.626 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:58 smithi154 ceph-mon[32578]: Reconfiguring daemon mon.a on smithi012 2024-04-26T21:52:58.626 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:52:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:58.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:58 smithi077 ceph-mon[32280]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:52:58.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:58 smithi077 ceph-mon[32280]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:52:58.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:58 smithi077 ceph-mon[32280]: Reconfiguring daemon mon.a on smithi012 2024-04-26T21:52:58.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:52:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- 172.21.15.154:0/3288624097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0101e50 msgr2=0x7f1ad0102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 --2- 172.21.15.154:0/3288624097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0101e50 0x7f1ad0102270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009910 tx=0x7f1ac0009c20 comp rx=0 tx=0).stop 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- 172.21.15.154:0/3288624097 shutdown_connections 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 --2- 172.21.15.154:0/3288624097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0101e50 0x7f1ad0102270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- 172.21.15.154:0/3288624097 >> 172.21.15.154:0/3288624097 conn(0x7f1ad00fd460 msgr2=0x7f1ad00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- 172.21.15.154:0/3288624097 shutdown_connections 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- 172.21.15.154:0/3288624097 wait complete. 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 Processor -- start 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.928+0000 7f1ad679d700 1 -- start start 2024-04-26T21:52:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad0101e50 0x7f1ad0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad01990c0 0x7f1ad01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0107d50 con 0x7f1ad01990c0 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0107bd0 con 0x7f1ad0101e50 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0107ed0 con 0x7f1ad0198c40 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57462/0 (socket says 172.21.15.154:57462) 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 -- 172.21.15.154:0/1747108300 learned_addr learned my addr 172.21.15.154:0/1747108300 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acffff700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad0101e50 0x7f1ad0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 -- 172.21.15.154:0/1747108300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad0101e50 msgr2=0x7f1ad0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad0101e50 0x7f1ad0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 -- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad01990c0 msgr2=0x7f1ad01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad01990c0 0x7f1ad01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 -- 172.21.15.154:0/1747108300 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a0b10 con 0x7f1ad0198c40 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acf7fe700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400b780 tx=0x7f1ac400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004230 con 0x7f1ad0198c40 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac40043d0 con 0x7f1ad0198c40 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a0e00 con 0x7f1ad0198c40 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.929+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4011b20 con 0x7f1ad0198c40 2024-04-26T21:52:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.930+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a1390 con 0x7f1ad0198c40 2024-04-26T21:52:59.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.930+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ac4011cc0 con 0x7f1ad0198c40 2024-04-26T21:52:59.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.930+0000 7f1acd7fa700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 0x7f1ab803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.930+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ac4003b10 con 0x7f1ad0198c40 2024-04-26T21:52:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.931+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad0064a80 con 0x7f1ad0198c40 2024-04-26T21:52:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.931+0000 7f1acffff700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 0x7f1ab803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.932+0000 7f1acffff700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 0x7f1ab803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005190 tx=0x7f1ac001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:52:59.936+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ac401fad0 con 0x7f1ad0198c40 2024-04-26T21:53:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.144+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi077=y;smithi154=z", "target": ["mon-mgr", ""]}) -- 0x7f1ad0063860 con 0x7f1ab8039ae0 2024-04-26T21:53:00.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.157+0000 7f1acd7fa700 1 -- 172.21.15.154:0/1747108300 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1ad0063860 con 0x7f1ab8039ae0 2024-04-26T21:53:00.157 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mgr update... 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 msgr2=0x7f1ab803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 0x7f1ab803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005190 tx=0x7f1ac001a040 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 msgr2=0x7f1ad019dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400b780 tx=0x7f1ac400ba90 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 shutdown_connections 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad01990c0 0x7f1ad01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad0101e50 0x7f1ad0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ab8039ae0 0x7f1ab803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 --2- 172.21.15.154:0/1747108300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad0198c40 0x7f1ad019dcb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 >> 172.21.15.154:0/1747108300 conn(0x7f1ad00fd460 msgr2=0x7f1ad006bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 shutdown_connections 2024-04-26T21:53:00.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:53:00.159+0000 7f1ad679d700 1 -- 172.21.15.154:0/1747108300 wait complete. 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:00 smithi154 ceph-mon[32578]: Reconfiguring daemon mon.b on smithi077 2024-04-26T21:53:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:53:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:00 smithi077 ceph-mon[32280]: Reconfiguring daemon mon.b on smithi077 2024-04-26T21:53:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:53:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:00 smithi012 ceph-mon[28184]: Reconfiguring daemon mon.b on smithi077 2024-04-26T21:53:00.806 DEBUG:teuthology.orchestra.run.smithi077:mgr.y> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:53:00.809 DEBUG:teuthology.orchestra.run.smithi154:mgr.z> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:53:00.811 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:53:00.811 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T21:53:00.811 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:53:00.874 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:53:00.874 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 21:52:00.091875411 +0000 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 21:46:50.500106862 +0000 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 21:46:50.500106862 +0000 2024-04-26T21:53:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-26 21:46:50.500106862 +0000 2024-04-26T21:53:00.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:53:01.018 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T21:53:01.018 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T21:53:01.019 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000395146 s, 1.3 MB/s 2024-04-26T21:53:01.022 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:53:01.081 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 21:52:00.154874895 +0000 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 21:46:51.075102412 +0000 2024-04-26T21:53:01.132 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 21:46:51.075102412 +0000 2024-04-26T21:53:01.133 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-26 21:46:51.075102412 +0000 2024-04-26T21:53:01.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:53:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T21:53:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T21:53:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000358946 s, 1.4 MB/s 2024-04-26T21:53:01.217 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:53:01.275 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi077=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: Saving service mgr spec with placement smithi012=x;smithi077=y;smithi154=z;count:3 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:01.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:01.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:01 smithi012 ceph-mon[28184]: Reconfiguring daemon mon.c on smithi154 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 21:52:00.222874338 +0000 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 21:46:51.703097552 +0000 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 21:46:51.703097552 +0000 2024-04-26T21:53:01.328 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-26 21:46:51.703097552 +0000 2024-04-26T21:53:01.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi077=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: Saving service mgr spec with placement smithi012=x;smithi077=y;smithi154=z;count:3 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:01.406 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:01 smithi154 ceph-mon[32578]: Reconfiguring daemon mon.c on smithi154 2024-04-26T21:53:01.411 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T21:53:01.411 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T21:53:01.411 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000387223 s, 1.3 MB/s 2024-04-26T21:53:01.415 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:53:01.474 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-04-26T21:53:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi077=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: Saving service mgr spec with placement smithi012=x;smithi077=y;smithi154=z;count:3 2024-04-26T21:53:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:01.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:53:01.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:53:01.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:53:01.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:01.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:01 smithi077 ceph-mon[32280]: Reconfiguring daemon mon.c on smithi154 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 21:52:00.301873691 +0000 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 21:46:52.309092861 +0000 2024-04-26T21:53:01.528 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 21:46:52.309092861 +0000 2024-04-26T21:53:01.529 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-26 21:46:52.309092861 +0000 2024-04-26T21:53:01.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:53:01.608 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T21:53:01.608 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T21:53:01.609 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000376306 s, 1.4 MB/s 2024-04-26T21:53:01.612 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:53:01.667 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T21:53:01.667 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:53:01.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:53:01.686 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 21:52:48.025111804 +0000 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 21:46:13.208166817 +0000 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 21:46:13.208166817 +0000 2024-04-26T21:53:01.745 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 21:46:13.208166817 +0000 2024-04-26T21:53:01.746 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:53:01.812 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T21:53:01.812 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T21:53:01.812 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000230692 s, 2.2 MB/s 2024-04-26T21:53:01.813 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:53:01.872 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-04-26T21:53:01.929 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 21:52:48.025111804 +0000 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 21:46:13.547160025 +0000 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 21:46:13.547160025 +0000 2024-04-26T21:53:01.930 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 21:46:13.547160025 +0000 2024-04-26T21:53:01.930 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:53:01.995 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T21:53:01.995 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T21:53:01.996 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000249078 s, 2.1 MB/s 2024-04-26T21:53:01.997 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:53:02.055 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-04-26T21:53:02.111 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 21:52:48.025111804 +0000 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 21:46:13.880153352 +0000 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 21:46:13.880153352 +0000 2024-04-26T21:53:02.112 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 21:46:13.880153352 +0000 2024-04-26T21:53:02.112 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:53:02.176 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T21:53:02.176 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T21:53:02.176 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000213911 s, 2.4 MB/s 2024-04-26T21:53:02.178 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:53:02.235 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 21:52:48.026111785 +0000 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 21:46:14.209146760 +0000 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 21:46:14.209146760 +0000 2024-04-26T21:53:02.291 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 21:46:14.209146760 +0000 2024-04-26T21:53:02.292 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:53:02.354 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T21:53:02.354 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T21:53:02.355 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000200437 s, 2.6 MB/s 2024-04-26T21:53:02.356 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:53:02.413 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-26T21:53:02.413 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:53:02.429 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:53:02.429 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-04-26T21:53:02.485 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:53:02.485 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:02.485 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T21:53:02.485 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:02.485 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:02.486 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-26 21:52:38.441182030 +0000 2024-04-26T21:53:02.486 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-26 21:45:33.685419433 +0000 2024-04-26T21:53:02.486 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-26 21:45:33.685419433 +0000 2024-04-26T21:53:02.486 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-26 21:45:33.685419433 +0000 2024-04-26T21:53:02.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:53:02.549 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-26T21:53:02.549 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-26T21:53:02.549 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.00021232 s, 2.4 MB/s 2024-04-26T21:53:02.550 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:53:02.606 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-26 21:52:38.441182030 +0000 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-26 21:45:34.027409999 +0000 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-26 21:45:34.027409999 +0000 2024-04-26T21:53:02.662 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-26 21:45:34.027409999 +0000 2024-04-26T21:53:02.663 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:53:02.725 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-26T21:53:02.725 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-26T21:53:02.725 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000226596 s, 2.3 MB/s 2024-04-26T21:53:02.726 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:53:02.783 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-26 21:52:38.442182003 +0000 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-26 21:45:34.362400759 +0000 2024-04-26T21:53:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-26 21:45:34.362400759 +0000 2024-04-26T21:53:02.839 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-26 21:45:34.362400759 +0000 2024-04-26T21:53:02.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:02.901 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-26T21:53:02.901 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-26T21:53:02.901 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000204631 s, 2.5 MB/s 2024-04-26T21:53:02.903 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:53:02.959 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-04-26T21:53:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:02.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:02.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:02.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:03.013 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-26 21:52:38.443181976 +0000 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-26 21:45:34.680391987 +0000 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-26 21:45:34.680391987 +0000 2024-04-26T21:53:03.014 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-26 21:45:34.680391987 +0000 2024-04-26T21:53:03.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:53:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:03.076 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-26T21:53:03.076 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-26T21:53:03.077 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000203865 s, 2.5 MB/s 2024-04-26T21:53:03.078 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:53:03.134 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-04-26T21:53:03.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:53:03.583 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:03.946 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:03 smithi154 ceph-mon[32578]: Deploying daemon mgr.z on smithi154 2024-04-26T21:53:03.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:03 smithi077 ceph-mon[32280]: Deploying daemon mgr.z on smithi154 2024-04-26T21:53:04.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:03 smithi012 ceph-mon[28184]: Deploying daemon mgr.z on smithi154 2024-04-26T21:53:04.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 podman[34428]: 2024-04-26 21:53:04.525562211 +0000 UTC m=+0.124235294 container create d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:53:04.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:04 smithi012 ceph-mon[28184]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:04 smithi154 ceph-mon[32578]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 podman[34428]: 2024-04-26 21:53:04.748753296 +0000 UTC m=+0.347426375 container init d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 podman[34428]: 2024-04-26 21:53:04.751448273 +0000 UTC m=+0.350121360 container start d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 ceph-mgr[34447]: -- 172.21.15.154:0/4584358 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe7f23a5a0 con 0x55fe7f20f000 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 bash[34428]: d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 systemd[1]: Started Ceph mgr.z for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:53:04.907 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:04.906+0000 7f7a9a159200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:53:04.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:04 smithi077 ceph-mon[32280]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:05.192 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:05.091+0000 7f7a9a159200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:53:05.192 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:05.192+0000 7f7a9a159200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:53:05.878 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:05.502+0000 7f7a9a159200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:53:05.878 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:05.586+0000 7f7a9a159200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:53:06.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: Deploying daemon mgr.y on smithi077 2024-04-26T21:53:06.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:05 smithi154 ceph-mon[32578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:06.153 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:06.154 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: Deploying daemon mgr.y on smithi077 2024-04-26T21:53:06.154 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:05 smithi077 ceph-mon[32280]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:06.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:06.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:06.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:53:06.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:06.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:06.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: Deploying daemon mgr.y on smithi077 2024-04-26T21:53:06.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:05 smithi012 ceph-mon[28184]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324021 s, 324 MB/s 2024-04-26T21:53:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-26T21:53:07.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:06.831+0000 7f7a9a159200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:53:07.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.003+0000 7f7a9a159200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:53:07.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-04-26T21:53:07.288 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 ceph-mgr[33682]: -- 172.21.15.77:0/1853440295 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f7f3665a0 con 0x560f7f33ac00 2024-04-26T21:53:07.461 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.174+0000 7f7a9a159200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:53:07.461 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.460+0000 7f7a9a159200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:53:07.551 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 bash[33662]: 23cae74151fb209275051c1c693713cf10c247d136fc563214edcbdf1a5d458b 2024-04-26T21:53:07.552 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 systemd[1]: Started Ceph mgr.y for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:53:07.552 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:07.367+0000 7f79d41f4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:53:07.552 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:07.551+0000 7f79d41f4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:53:07.740 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:07.764 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.552+0000 7f7a9a159200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:53:07.764 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.764+0000 7f7a9a159200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:53:07.978 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:07.654+0000 7f79d41f4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:53:07.978 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:07.977+0000 7f79d41f4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:53:08.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:07.964+0000 7f7a9a159200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:53:08.232 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:08.065+0000 7f79d41f4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:53:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:08.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:08.616 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.662 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:08.662 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.662 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.662 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.662 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.663 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:08.663 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:08.891 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:08.615+0000 7f7a9a159200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:53:09.360 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:09.056+0000 7f7a9a159200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:53:09.360 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:09.155+0000 7f7a9a159200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:53:09.360 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:09.359+0000 7f7a9a159200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:53:09.625 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:09.293+0000 7f79d41f4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:53:09.626 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:09.458+0000 7f79d41f4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:53:09.626 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:09.625+0000 7f79d41f4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:53:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 -- 172.21.15.12:0/321373397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc104690 msgr2=0x7fa6fc104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 --2- 172.21.15.12:0/321373397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc104690 0x7fa6fc104af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80099c0 tx=0x7fa6e8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 -- 172.21.15.12:0/321373397 shutdown_connections 2024-04-26T21:53:09.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 --2- 172.21.15.12:0/321373397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc105030 0x7fa6fc10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 --2- 172.21.15.12:0/321373397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc103490 0x7fa6fc1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 --2- 172.21.15.12:0/321373397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc104690 0x7fa6fc104af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 -- 172.21.15.12:0/321373397 >> 172.21.15.12:0/321373397 conn(0x7fa6fc0fe9f0 msgr2=0x7fa6fc100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 -- 172.21.15.12:0/321373397 shutdown_connections 2024-04-26T21:53:09.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.691+0000 7fa704000700 1 -- 172.21.15.12:0/321373397 wait complete. 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.692+0000 7fa704000700 1 Processor -- start 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.692+0000 7fa704000700 1 -- start start 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.692+0000 7fa704000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa704000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc104690 0x7fa6fc19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa704000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc105030 0x7fa6fc1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa704000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10ef80 con 0x7fa6fc104690 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa704000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10ee00 con 0x7fa6fc103490 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa704000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10f100 con 0x7fa6fc105030 2024-04-26T21:53:09.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:55122/0 (socket says 172.21.15.12:55122) 2024-04-26T21:53:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 -- 172.21.15.12:0/783159530 learned_addr learned my addr 172.21.15.12:0/783159530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:53:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa700c00700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc104690 0x7fa6fc19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa702000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc105030 0x7fa6fc1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 -- 172.21.15.12:0/783159530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc105030 msgr2=0x7fa6fc1a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc105030 0x7fa6fc1a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 -- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc104690 msgr2=0x7fa6fc19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc104690 0x7fa6fc19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.693+0000 7fa701600700 1 -- 172.21.15.12:0/783159530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1a34a0 con 0x7fa6fc103490 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa701600700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e950 tx=0x7fa6f000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000eea0 con 0x7fa6fc103490 2024-04-26T21:53:09.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f00044e0 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0003da0 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc1a3790 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.694+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1a3c80 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.695+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa6f0021440 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.696+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc06a110 con 0x7fa6fc103490 2024-04-26T21:53:09.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.696+0000 7fa6fa000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 0x7fa6e403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.696+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6f004cd70 con 0x7fa6fc103490 2024-04-26T21:53:09.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.696+0000 7fa700c00700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 0x7fa6e403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.697+0000 7fa700c00700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 0x7fa6e403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:09.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.701+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6f001f7e0 con 0x7fa6fc103490 2024-04-26T21:53:09.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:09.950+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa6fc076d70 con 0x7fa6e4039890 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:09.890+0000 7f7a9a159200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:09.980+0000 7f7a9a159200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: Reconfiguring daemon mgr.x on smithi012 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:10.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.196 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:09.908+0000 7f79d41f4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:09.998+0000 7f79d41f4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:10.196+0000 7f79d41f4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: Reconfiguring daemon mgr.x on smithi012 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:10.198 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:10.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: Reconfiguring daemon mgr.x on smithi012 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:10.237 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:10 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:10.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:10.385+0000 7f79d41f4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:53:10.564 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:10.222+0000 7f7a9a159200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:53:10.564 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:10.354+0000 7f7a9a159200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:53:10.564 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:10.450+0000 7f7a9a159200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:53:10.565 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:10.564+0000 7f7a9a159200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:53:11.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:11 smithi154 ceph-mon[32578]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:11.141 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:10.877+0000 7f7a9a159200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:53:11.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:11 smithi077 ceph-mon[32280]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:11.287 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:10.971+0000 7f79d41f4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:53:11.288 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:11 smithi012 ceph-mon[28184]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:11.676 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:11.382+0000 7f79d41f4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:53:11.676 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:11.476+0000 7f79d41f4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:53:11.676 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:11.675+0000 7f79d41f4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:53:11.891 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:11.479+0000 7f7a9a159200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:53:11.891 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 21:53:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T21:53:11.698+0000 7f7a9a159200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:53:12.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:12.460+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fa6f0007d10 con 0x7fa6fc103490 2024-04-26T21:53:12.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.192+0000 7f79d41f4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:53:12.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.282+0000 7f79d41f4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: Standby manager daemon z started 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:12.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:53:12.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:12.533 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:12.533 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.533 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: Standby manager daemon z started 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:53:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:12 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:12.853 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.520+0000 7f79d41f4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:53:12.853 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.649+0000 7f79d41f4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:53:12.853 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.741+0000 7f79d41f4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:53:12.854 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:12.853+0000 7f79d41f4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: Standby manager daemon z started 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:53:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:12.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:53:12.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:12 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/2607477094' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:13.462 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:13.147+0000 7f79d41f4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:53:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:13 smithi077 ceph-mon[32280]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T21:53:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:13 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:53:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:13 smithi012 ceph-mon[28184]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T21:53:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:13 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:53:13.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:13 smithi154 ceph-mon[32578]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T21:53:13.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:13 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:53:14.232 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:13.740+0000 7f79d41f4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:53:14.232 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 21:53:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T21:53:13.951+0000 7f79d41f4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:53:14.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:14.487+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mon.2 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa6f0010520 con 0x7fa6fc103490 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: Standby manager daemon y started 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]: dispatch 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]': finished 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:53:14.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:14 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: Standby manager daemon y started 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:53:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:14.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]: dispatch 2024-04-26T21:53:14.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]': finished 2024-04-26T21:53:14.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:53:14.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:14 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: Standby manager daemon y started 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/12705522' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]: dispatch 2024-04-26T21:53:14.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2475047426' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26c2f4bf-8ef4-4f89-9dc8-b00f6e556980"}]': finished 2024-04-26T21:53:14.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:53:14.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:14 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:15.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:15 smithi012 ceph-mon[28184]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T21:53:15.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:53:15.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:15 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2142527774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:15 smithi154 ceph-mon[32578]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T21:53:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:53:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:15 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2142527774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:15 smithi077 ceph-mon[32280]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T21:53:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:53:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:15 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2142527774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:16.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:16 smithi012 ceph-mon[28184]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:16 smithi154 ceph-mon[32578]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:16.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:16 smithi077 ceph-mon[32280]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:18.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:18 smithi012 ceph-mon[28184]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:18.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:18 smithi154 ceph-mon[32578]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:18 smithi077 ceph-mon[32280]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:20.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:20 smithi012 ceph-mon[28184]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:20 smithi154 ceph-mon[32578]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:20.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:20 smithi077 ceph-mon[32280]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:22.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:22 smithi012 ceph-mon[28184]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:22.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:22 smithi154 ceph-mon[32578]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:22.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:22 smithi077 ceph-mon[32280]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:24.764 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:24 smithi012 ceph-mon[28184]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:24.764 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:53:24.764 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:24.764 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:24 smithi012 ceph-mon[28184]: Deploying daemon osd.0 on smithi012 2024-04-26T21:53:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:24 smithi154 ceph-mon[32578]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:53:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:24 smithi154 ceph-mon[32578]: Deploying daemon osd.0 on smithi012 2024-04-26T21:53:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:24 smithi077 ceph-mon[32280]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:53:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:24 smithi077 ceph-mon[32280]: Deploying daemon osd.0 on smithi012 2024-04-26T21:53:26.818 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:26 smithi012 ceph-mon[28184]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:26.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:26 smithi154 ceph-mon[32578]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:26 smithi077 ceph-mon[32280]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:28 smithi012 ceph-mon[28184]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:28 smithi154 ceph-mon[32578]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:28.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:28 smithi077 ceph-mon[32280]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:30.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:30 smithi012 ceph-mon[28184]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:30.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:30 smithi154 ceph-mon[32578]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:30.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:30 smithi077 ceph-mon[32280]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:33.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:32 smithi012 ceph-mon[28184]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:33.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:33.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:33.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:33.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:32 smithi154 ceph-mon[32578]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:33.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:33.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:33.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:32 smithi077 ceph-mon[32280]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:34.930 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:34 smithi012 ceph-mon[28184]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:35.034 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:34 smithi077 ceph-mon[32280]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:34 smithi154 ceph-mon[32578]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:36.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:36.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:36.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:36.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:36.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:36 smithi012 ceph-mon[28184]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:36.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:36 smithi154 ceph-mon[32578]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:36.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.696+0000 7fa6fa000700 1 -- 172.21.15.12:0/783159530 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6fc076d70 con 0x7fa6e4039890 2024-04-26T21:53:36.697 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-04-26T21:53:36.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 msgr2=0x7fa6e403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:36.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 0x7fa6e403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e8005fb0 comp rx=0 tx=0).stop 2024-04-26T21:53:36.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 msgr2=0x7fa6fc19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:36.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e950 tx=0x7fa6f000ec60 comp rx=0 tx=0).stop 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 shutdown_connections 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa6e4039890 0x7fa6e403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6fc105030 0x7fa6fc1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6fc103490 0x7fa6fc19b840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 --2- 172.21.15.12:0/783159530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa6fc104690 0x7fa6fc19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.700+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 >> 172.21.15.12:0/783159530 conn(0x7fa6fc0fe9f0 msgr2=0x7fa6fc075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.701+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 shutdown_connections 2024-04-26T21:53:36.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:53:36.701+0000 7fa704000700 1 -- 172.21.15.12:0/783159530 wait complete. 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:36.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:36.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:36 smithi077 ceph-mon[32280]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:53:37.517 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:53:37.521 INFO:tasks.cephadm:Deploying osd.1 on smithi077 with /dev/vg_nvme/lv_4... 2024-04-26T21:53:37.522 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:53:37.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:37.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-mon[28184]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:37.765 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:53:37.886 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:37.887 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:37 smithi077 ceph-mon[32280]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:37.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:37 smithi154 ceph-mon[32578]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:53:38.074 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0[43289]: 2024-04-26T21:53:37.809+0000 7fd1af800700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:53:38.074 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:53:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0[43289]: 2024-04-26T21:53:37.819+0000 7fd1aa600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:53:38.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:38 smithi012 ceph-mon[28184]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:38.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:38 smithi012 ceph-mon[28184]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:53:38.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:38 smithi012 ceph-mon[28184]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:53:38.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:38.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:39.024 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:38 smithi077 ceph-mon[32280]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:39.025 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:38 smithi077 ceph-mon[32280]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:53:39.025 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:38 smithi077 ceph-mon[32280]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:53:39.025 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:39.025 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:38 smithi154 ceph-mon[32578]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:38 smithi154 ceph-mon[32578]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:53:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:38 smithi154 ceph-mon[32578]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:53:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:39.592 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:53:39.592 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:53:39.592 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T21:53:39.592 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T21:53:39.592 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115402 s, 909 MB/s 2024-04-26T21:53:39.593 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T21:53:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:39 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:53:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:39 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:53:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:39 smithi012 ceph-mon[28184]: osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] boot 2024-04-26T21:53:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:39 smithi012 ceph-mon[28184]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:53:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:39 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:40.121 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:39 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:53:40.121 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:39 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:53:40.121 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:39 smithi077 ceph-mon[32280]: osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] boot 2024-04-26T21:53:40.121 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:39 smithi077 ceph-mon[32280]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:53:40.121 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:39 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:40.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:39 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:53:40.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:39 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:53:40.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:39 smithi154 ceph-mon[32578]: osd.0 [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] boot 2024-04-26T21:53:40.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:39 smithi154 ceph-mon[32578]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:53:40.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:39 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:53:40.146 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-04-26T21:53:40.378 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:53:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:40 smithi012 ceph-mon[28184]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:40 smithi012 ceph-mon[28184]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:53:41.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:40 smithi154 ceph-mon[32578]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:40 smithi154 ceph-mon[32578]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:53:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:40 smithi077 ceph-mon[32280]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:53:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:40 smithi077 ceph-mon[32280]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 -- 172.21.15.77:0/3927690234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510103f20 msgr2=0x7f9510106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 --2- 172.21.15.77:0/3927690234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510103f20 0x7f9510106310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9500009990 tx=0x7f9500009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 -- 172.21.15.77:0/3927690234 shutdown_connections 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 --2- 172.21.15.77:0/3927690234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510109180 0x7f951010b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 --2- 172.21.15.77:0/3927690234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510103f20 0x7f9510106310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 --2- 172.21.15.77:0/3927690234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510106850 0x7f9510108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 -- 172.21.15.77:0/3927690234 >> 172.21.15.77:0/3927690234 conn(0x7f95100fd950 msgr2=0x7f95100ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 -- 172.21.15.77:0/3927690234 shutdown_connections 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.854+0000 7f951629b700 1 -- 172.21.15.77:0/3927690234 wait complete. 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 Processor -- start 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 -- start start 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510103f20 0x7f95101996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510106850 0x7f9510199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f951010ef90 con 0x7f9510103f20 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f951010ee10 con 0x7f9510109180 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f951629b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f951010f110 con 0x7f9510106850 2024-04-26T21:53:41.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46030/0 (socket says 172.21.15.77:46030) 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 -- 172.21.15.77:0/1404820016 learned_addr learned my addr 172.21.15.77:0/1404820016 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f950f7fe700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510106850 0x7f9510199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 -- 172.21.15.77:0/1404820016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510106850 msgr2=0x7f9510199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510106850 0x7f9510199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 -- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510103f20 msgr2=0x7f95101996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510103f20 0x7f95101996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.855+0000 7f9514838700 1 -- 172.21.15.77:0/1404820016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95101a1300 con 0x7f9510109180 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f9514838700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f950800ada0 tx=0x7f950800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508017020 con 0x7f9510109180 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95101a15f0 con 0x7f9510109180 2024-04-26T21:53:41.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f950800ce30 con 0x7f9510109180 2024-04-26T21:53:41.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080053a0 con 0x7f9510109180 2024-04-26T21:53:41.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.856+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a1b60 con 0x7f9510109180 2024-04-26T21:53:41.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.857+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951006a110 con 0x7f9510109180 2024-04-26T21:53:41.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.858+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f950800c980 con 0x7f9510109180 2024-04-26T21:53:41.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.858+0000 7f950d7fa700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 0x7f94f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.859+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f95080ce5a0 con 0x7f9510109180 2024-04-26T21:53:41.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.859+0000 7f950ffff700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 0x7f94f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.860+0000 7f950ffff700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 0x7f94f80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95000098e0 tx=0x7f9500003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:41.862+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9508021020 con 0x7f9510109180 2024-04-26T21:53:42.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:53:42.072+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9510063860 con 0x7f94f80a3bb0 2024-04-26T21:53:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:43.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:53:43.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:43.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:43.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:43.929 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:43 smithi077 ceph-mon[32280]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:43.929 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:43 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi012 2024-04-26T21:53:43.929 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:43 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 17094M 2024-04-26T21:53:44.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:43 smithi154 ceph-mon[32578]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:44.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:43 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi012 2024-04-26T21:53:44.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:43 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 17094M 2024-04-26T21:53:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:43 smithi012 ceph-mon[28184]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:43 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi012 2024-04-26T21:53:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:43 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 17094M 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/1671521495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]': finished 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:53:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/1671521495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]': finished 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:53:45.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/1671521495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]: dispatch 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f493e866-faee-4d6e-a462-c582971cf31b"}]': finished 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:53:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:53:45.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:45 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/965741094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:46.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:45 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/965741094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:46.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:45 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/965741094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:53:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:46 smithi154 ceph-mon[32578]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:47.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:46 smithi077 ceph-mon[32280]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:47.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:46 smithi012 ceph-mon[28184]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:48.214 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:47 smithi077 ceph-mon[32280]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:48.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:47 smithi012 ceph-mon[28184]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:47 smithi154 ceph-mon[32578]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:50.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:50 smithi012 ceph-mon[28184]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:50.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:50 smithi077 ceph-mon[32280]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:50.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:50 smithi154 ceph-mon[32578]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:52.366 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:52 smithi077 ceph-mon[32280]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:52.366 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:53:52.366 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:52.366 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:52 smithi077 ceph-mon[32280]: Deploying daemon osd.1 on smithi077 2024-04-26T21:53:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:52 smithi154 ceph-mon[32578]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:53:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:52 smithi154 ceph-mon[32578]: Deploying daemon osd.1 on smithi077 2024-04-26T21:53:52.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:52 smithi012 ceph-mon[28184]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:52.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:53:52.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:52.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:52 smithi012 ceph-mon[28184]: Deploying daemon osd.1 on smithi077 2024-04-26T21:53:54.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:54 smithi154 ceph-mon[32578]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:54.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:54 smithi077 ceph-mon[32280]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:54.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:54 smithi012 ceph-mon[28184]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:56.090 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:56 smithi077 ceph-mon[32280]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:56.347 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:56 smithi012 ceph-mon[28184]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:56.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:56 smithi154 ceph-mon[32578]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:58.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:58.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:53:57 smithi077 ceph-mon[32280]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:58.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:58.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:53:57 smithi012 ceph-mon[28184]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:53:58.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:53:58.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:58.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:53:57 smithi154 ceph-mon[32578]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.617+0000 7f950d7fa700 1 -- 172.21.15.77:0/1404820016 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9510063860 con 0x7f94f80a3bb0 2024-04-26T21:54:00.618 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 1 on host 'smithi077' 2024-04-26T21:54:00.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 msgr2=0x7f94f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 0x7f94f80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95000098e0 tx=0x7f9500003860 comp rx=0 tx=0).stop 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 msgr2=0x7f95101a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f950800ada0 tx=0x7f950800c330 comp rx=0 tx=0).stop 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 shutdown_connections 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9510109180 0x7f95101a0c60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510103f20 0x7f95101996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f94f80a3bb0 0x7f94f80a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 --2- 172.21.15.77:0/1404820016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510106850 0x7f9510199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 >> 172.21.15.77:0/1404820016 conn(0x7f95100fd950 msgr2=0x7f9510104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:00.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 shutdown_connections 2024-04-26T21:54:00.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:54:00.619+0000 7f951629b700 1 -- 172.21.15.77:0/1404820016 wait complete. 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.211 DEBUG:teuthology.orchestra.run.smithi077:osd.1> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:54:01.213 INFO:tasks.cephadm:Deploying osd.2 on smithi154 with /dev/vg_nvme/lv_4... 2024-04-26T21:54:01.214 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:54:01.459 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:54:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:01.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:01 smithi077 ceph-mon[32280]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:01.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:01 smithi077 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:01.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:01 smithi012 ceph-mon[28184]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:01.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:01 smithi012 ceph-mon[28184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:01.878 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.878 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:01.878 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:01.878 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:01 smithi154 ceph-mon[32578]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:01.878 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:01 smithi154 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:54:02.767 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:02.767 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:54:02.768 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:54:02.768 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:02.768 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:02.768 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:02 smithi154 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:54:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:54:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:02 smithi077 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:54:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:54:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: from='osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:03.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:02 smithi012 ceph-mon[28184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117866 s, 890 MB/s 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-26T21:54:03.597 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1[36868]: 2024-04-26T21:54:03.294+0000 7fe566831700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:54:03.597 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1[36868]: 2024-04-26T21:54:03.299+0000 7fe55e61c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:54:03.856 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:54:03.856 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-mon[32280]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:54:03.856 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:03.856 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:03.856 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:03 smithi077 ceph-mon[32280]: from='osd.1 ' entity='osd.1' 2024-04-26T21:54:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:03 smithi154 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:54:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:03 smithi154 ceph-mon[32578]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:54:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:03 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:03.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:03 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:03.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:03 smithi154 ceph-mon[32578]: from='osd.1 ' entity='osd.1' 2024-04-26T21:54:03.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-04-26T21:54:04.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:03 smithi012 ceph-mon[28184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:54:04.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:03 smithi012 ceph-mon[28184]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:54:04.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:04.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:03 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:04.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:03 smithi012 ceph-mon[28184]: from='osd.1 ' entity='osd.1' 2024-04-26T21:54:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] boot 2024-04-26T21:54:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:54:04.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:04.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] boot 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:54:04.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:05.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: osd.1 [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] boot 2024-04-26T21:54:05.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:54:05.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- 172.21.15.154:0/2151034922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 msgr2=0x7fcfb81027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 --2- 172.21.15.154:0/2151034922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81027c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009990 tx=0x7fcfa8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- 172.21.15.154:0/2151034922 shutdown_connections 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 --2- 172.21.15.154:0/2151034922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfb8103f40 0x7fcfb810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 --2- 172.21.15.154:0/2151034922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81027c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 --2- 172.21.15.154:0/2151034922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 0x7fcfb8103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- 172.21.15.154:0/2151034922 >> 172.21.15.154:0/2151034922 conn(0x7fcfb80fd980 msgr2=0x7fcfb80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- 172.21.15.154:0/2151034922 shutdown_connections 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- 172.21.15.154:0/2151034922 wait complete. 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 Processor -- start 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.687+0000 7fcfbe885700 1 -- start start 2024-04-26T21:54:05.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 0x7fcfb8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfb8103f40 0x7fcfb819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810df10 con 0x7fcfb8103f40 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810dd90 con 0x7fcfb81035a0 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810e090 con 0x7fcfb81023a0 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36380/0 (socket says 172.21.15.154:36380) 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 -- 172.21.15.154:0/3773013308 learned_addr learned my addr 172.21.15.154:0/3773013308 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb77fe700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 0x7fcfb8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 -- 172.21.15.154:0/3773013308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 msgr2=0x7fcfb8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 0x7fcfb8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 -- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfb8103f40 msgr2=0x7fcfb819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfb8103f40 0x7fcfb819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 -- 172.21.15.154:0/3773013308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81a0240 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb7fff700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80047b0 tx=0x7fcfa8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8027070 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8022470 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb81a04d0 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.688+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800f6c0 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.689+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb81a09a0 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.690+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb806a110 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.691+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fcfa802a070 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.691+0000 7fcfb57fa700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 0x7fcfa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.691+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fcfa80cfa40 con 0x7fcfb81023a0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.692+0000 7fcfb77fe700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 0x7fcfa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.693+0000 7fcfb77fe700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 0x7fcfa00a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009a10 tx=0x7fcfac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.696+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcfa8013070 con 0x7fcfb81023a0 2024-04-26T21:54:05.884 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:05 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi077 2024-04-26T21:54:05.885 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:05 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-26T21:54:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:05 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi077 2024-04-26T21:54:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:05 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-26T21:54:05.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:05.904+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcfb8063860 con 0x7fcfa00a3bb0 2024-04-26T21:54:06.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:05 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi077 2024-04-26T21:54:06.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:05 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 17088M 2024-04-26T21:54:06.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:06 smithi154 ceph-mon[32578]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:06.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:06 smithi154 ceph-mon[32578]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:54:06.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:06.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:06.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:06.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:06 smithi077 ceph-mon[32280]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:06.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:06 smithi077 ceph-mon[32280]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:54:06.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:06.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:06.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:06 smithi012 ceph-mon[28184]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:06 smithi012 ceph-mon[28184]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:54:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:07.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:07 smithi154 ceph-mon[32578]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:07.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:07 smithi077 ceph-mon[32280]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:08.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:07 smithi012 ceph-mon[28184]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/2645105516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]': finished 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:54:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/2645105516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]': finished 2024-04-26T21:54:08.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:54:08.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:09.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/2645105516' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]: dispatch 2024-04-26T21:54:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a38e236-f013-4ab3-9310-272f5ec1b653"}]': finished 2024-04-26T21:54:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:54:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:09.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:09 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/1890161931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:09 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/1890161931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:10.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:09 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/1890161931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:10.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:10 smithi154 ceph-mon[32578]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:10.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:10 smithi077 ceph-mon[32280]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:11.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:10 smithi012 ceph-mon[28184]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:12.794 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:12 smithi154 ceph-mon[32578]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:12 smithi077 ceph-mon[32280]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:12 smithi012 ceph-mon[28184]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:14.770 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:14 smithi154 ceph-mon[32578]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:14 smithi077 ceph-mon[32280]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:14 smithi012 ceph-mon[28184]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:15.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:54:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:15.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:54:15.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:54:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:16.820 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:16 smithi154 ceph-mon[32578]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:16.820 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:16 smithi154 ceph-mon[32578]: Deploying daemon osd.2 on smithi154 2024-04-26T21:54:16.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:16 smithi077 ceph-mon[32280]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:16.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:16 smithi077 ceph-mon[32280]: Deploying daemon osd.2 on smithi154 2024-04-26T21:54:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:16 smithi012 ceph-mon[28184]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:16 smithi012 ceph-mon[28184]: Deploying daemon osd.2 on smithi154 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:18 smithi154 ceph-mon[32578]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:18 smithi077 ceph-mon[32280]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:19.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:18 smithi012 ceph-mon[28184]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:20.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:20 smithi154 ceph-mon[32578]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:20.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:20 smithi077 ceph-mon[32280]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:20 smithi012 ceph-mon[28184]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:21 smithi154 ceph-mon[32578]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:21 smithi077 ceph-mon[32280]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:21 smithi012 ceph-mon[28184]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.490+0000 7fcfb57fa700 1 -- 172.21.15.154:0/3773013308 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcfb8063860 con 0x7fcfa00a3bb0 2024-04-26T21:54:24.490 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 2 on host 'smithi154' 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:24.491 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 msgr2=0x7fcfa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 0x7fcfa00a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009a10 tx=0x7fcfac009190 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 msgr2=0x7fcfb81985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80047b0 tx=0x7fcfa8009f50 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 shutdown_connections 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfb8103f40 0x7fcfb819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcfa00a3bb0 0x7fcfa00a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfb81023a0 0x7fcfb81985e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 --2- 172.21.15.154:0/3773013308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcfb81035a0 0x7fcfb8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 >> 172.21.15.154:0/3773013308 conn(0x7fcfb80fd980 msgr2=0x7fcfb8106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.492+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 shutdown_connections 2024-04-26T21:54:24.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:54:24.493+0000 7fcfbe885700 1 -- 172.21.15.154:0/3773013308 wait complete. 2024-04-26T21:54:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:24.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:24.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.085 DEBUG:teuthology.orchestra.run.smithi154:osd.2> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:54:25.087 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-04-26T21:54:25.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:54:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:25.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:25.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:25 smithi154 ceph-mon[32578]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:25.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:25 smithi154 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:25.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:25.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:25 smithi012 ceph-mon[28184]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:25.714 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:25 smithi012 ceph-mon[28184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:25 smithi077 ceph-mon[32280]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:25 smithi077 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:54:26.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:26.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:54:26.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:54:26.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:26.541 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:26.542 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:26 smithi154 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:54:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:54:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:26.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:26 smithi012 ceph-mon[28184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:54:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:54:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: from='osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:26 smithi077 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:54:27.498 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2[37745]: 2024-04-26T21:54:27.236+0000 7fd3c26b6700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:54:27.498 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2[37745]: 2024-04-26T21:54:27.241+0000 7fd3ba4a1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:54:27.715 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:27 smithi012 ceph-mon[28184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:54:27.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:27 smithi012 ceph-mon[28184]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:54:27.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:27 smithi012 ceph-mon[28184]: from='osd.2 ' entity='osd.2' 2024-04-26T21:54:27.778 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:54:27.778 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-mon[32578]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:54:27.778 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.779 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.779 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:27 smithi154 ceph-mon[32578]: from='osd.2 ' entity='osd.2' 2024-04-26T21:54:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:27 smithi077 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:54:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:27 smithi077 ceph-mon[32280]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:54:27.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:27.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:27 smithi077 ceph-mon[32280]: from='osd.2 ' entity='osd.2' 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0348554 s, 301 MB/s 2024-04-26T21:54:28.655 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-26T21:54:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:54:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:54:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] boot 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:54:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] boot 2024-04-26T21:54:28.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:54:28.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:28.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: osd.2 [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] boot 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:54:28.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:54:29.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-04-26T21:54:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:29 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi154 2024-04-26T21:54:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:29 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-26T21:54:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:29 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:29.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:29 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi154 2024-04-26T21:54:29.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:29 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-26T21:54:29.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:29 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:29.903 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:29 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi154 2024-04-26T21:54:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:29 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-26T21:54:29.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:29 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:30 smithi012 ceph-mon[28184]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:30 smithi012 ceph-mon[28184]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:54:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:30 smithi154 ceph-mon[32578]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:30 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:30 smithi154 ceph-mon[32578]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:54:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:30 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:30.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:30 smithi077 ceph-mon[32280]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:30.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:30 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:30.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:30 smithi077 ceph-mon[32280]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:54:30.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:30 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:54:31.481 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47645]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47645]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47645]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47648]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47648]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.482 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47648]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.555 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41008]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41008]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41008]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41011]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.556 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41011]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41818]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41821]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41824]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41827]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:31 smithi154 sudo[41827]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47658]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47658]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47671]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47671]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47671]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 sudo[47671]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:31 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:54:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:54:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41014]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41014]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41014]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41017]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41017]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41017]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:54:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:31 smithi077 sudo[41017]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:54:32.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 -- 172.21.15.12:0/2927009707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34581045a0 msgr2=0x7f3458106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 --2- 172.21.15.12:0/2927009707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34581045a0 0x7f3458106a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 -- 172.21.15.12:0/2927009707 shutdown_connections 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 --2- 172.21.15.12:0/2927009707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3458106f70 0x7f34581073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 --2- 172.21.15.12:0/2927009707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458101c30 0x7f3458104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 --2- 172.21.15.12:0/2927009707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34581045a0 0x7f3458106a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 -- 172.21.15.12:0/2927009707 >> 172.21.15.12:0/2927009707 conn(0x7f34580fd850 msgr2=0x7f34580ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:32.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 -- 172.21.15.12:0/2927009707 shutdown_connections 2024-04-26T21:54:32.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.142+0000 7f345ee00700 1 -- 172.21.15.12:0/2927009707 wait complete. 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 Processor -- start 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 -- start start 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 0x7f345819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 0x7f345819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458106f70 0x7f34581a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34581153f0 con 0x7f3458101c30 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3458115270 con 0x7f34581045a0 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.144+0000 7f345ee00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3458115570 con 0x7f3458106f70 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345da00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 0x7f345819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345da00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 0x7f345819fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35378/0 (socket says 172.21.15.12:35378) 2024-04-26T21:54:32.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345da00700 1 -- 172.21.15.12:0/2988534820 learned_addr learned my addr 172.21.15.12:0/2988534820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 0x7f345819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345e400700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458106f70 0x7f34581a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 -- 172.21.15.12:0/2988534820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458106f70 msgr2=0x7f34581a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458106f70 0x7f34581a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 -- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 msgr2=0x7f345819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 0x7f345819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.145+0000 7f345d000700 1 -- 172.21.15.12:0/2988534820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34581a76a0 con 0x7f34581045a0 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f345d000700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 0x7f345819ff80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f34440057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444026070 con 0x7f34581045a0 2024-04-26T21:54:32.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34581a7930 con 0x7f34581045a0 2024-04-26T21:54:32.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34581a7e90 con 0x7f34581045a0 2024-04-26T21:54:32.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3444021410 con 0x7f34581045a0 2024-04-26T21:54:32.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.146+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444017650 con 0x7f34581045a0 2024-04-26T21:54:32.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.147+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438005360 con 0x7f34581045a0 2024-04-26T21:54:32.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.149+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f3444029070 con 0x7f34581045a0 2024-04-26T21:54:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.151+0000 7f3456000700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 0x7f34400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.151+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f34440cfa00 con 0x7f34581045a0 2024-04-26T21:54:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.151+0000 7f345da00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 0x7f34400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.152+0000 7f345da00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 0x7f34400a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d20 tx=0x7f344c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:32.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.155+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3444013070 con 0x7f34581045a0 2024-04-26T21:54:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:32.398+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3438000bf0 con 0x7f34400a3c30 2024-04-26T21:54:32.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:32.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:32.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:32.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:32.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:54:32.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:54:32.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:54:32.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:54:32.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:54:32.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:33.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:33.579+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3444009d10 con 0x7f34581045a0 2024-04-26T21:54:33.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:33 smithi012 ceph-mon[28184]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:33.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:33 smithi154 ceph-mon[32578]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:33 smithi077 ceph-mon[32280]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:34.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:34 smithi012 ceph-mon[28184]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:34.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:34 smithi012 ceph-mon[28184]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:54:34.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:34 smithi154 ceph-mon[32578]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:34.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:34 smithi154 ceph-mon[32578]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:54:34.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:34 smithi077 ceph-mon[32280]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:34.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:34 smithi077 ceph-mon[32280]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3570482701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]': finished 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:54:36.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:36 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:54:36.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:36.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3570482701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]': finished 2024-04-26T21:54:36.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:54:36.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:36 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3570482701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]: dispatch 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d44a553b-4160-4233-b691-acd3104b7eac"}]': finished 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:54:36.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:36 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:54:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:37 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/271787634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:37.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:37 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/271787634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:38.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:37 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/271787634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:54:38.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:38 smithi154 ceph-mon[32578]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:38 smithi077 ceph-mon[32280]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:38 smithi012 ceph-mon[28184]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:40.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:40 smithi154 ceph-mon[32578]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:40.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:40 smithi077 ceph-mon[32280]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:41.070 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:40 smithi012 ceph-mon[28184]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:42.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:42 smithi012 ceph-mon[28184]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:42.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:42 smithi154 ceph-mon[32578]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:42 smithi077 ceph-mon[32280]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:44.765 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:44 smithi012 ceph-mon[28184]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:44 smithi154 ceph-mon[32578]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:44 smithi077 ceph-mon[32280]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:46 smithi154 ceph-mon[32578]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:54:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:46 smithi077 ceph-mon[32280]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:54:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:47.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:46 smithi012 ceph-mon[28184]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:47.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:54:47.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:47 smithi012 ceph-mon[28184]: Deploying daemon osd.3 on smithi012 2024-04-26T21:54:47.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:47 smithi154 ceph-mon[32578]: Deploying daemon osd.3 on smithi012 2024-04-26T21:54:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:47 smithi077 ceph-mon[32280]: Deploying daemon osd.3 on smithi012 2024-04-26T21:54:48.886 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:48 smithi012 ceph-mon[28184]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:48.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:48 smithi154 ceph-mon[32578]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:48 smithi077 ceph-mon[32280]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:50 smithi154 ceph-mon[32578]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:50.967 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:50 smithi012 ceph-mon[28184]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:50.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:50 smithi077 ceph-mon[32280]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:52.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:52 smithi012 ceph-mon[28184]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:52.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:52 smithi077 ceph-mon[32280]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:52 smithi154 ceph-mon[32578]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:54.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:54 smithi012 ceph-mon[28184]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:54.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:54 smithi077 ceph-mon[32280]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:55.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:54 smithi154 ceph-mon[32578]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:55.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:55 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:55.966 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:55 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:55.966 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:55 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:55 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:56.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:55 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:56.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:55 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:56.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:55 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:56 smithi077 ceph-mon[32280]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:56 smithi012 ceph-mon[28184]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:57.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:56 smithi154 ceph-mon[32578]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:54:58 smithi077 ceph-mon[32280]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:58.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:54:58 smithi012 ceph-mon[28184]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:54:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:54:58 smithi154 ceph-mon[32578]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:54:59.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.443+0000 7f3456000700 1 -- 172.21.15.12:0/2988534820 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3438000bf0 con 0x7f34400a3c30 2024-04-26T21:54:59.444 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 msgr2=0x7f34400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 0x7f34400a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d20 tx=0x7f344c009260 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 msgr2=0x7f345819ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 0x7f345819ff80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f34440057a0 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 shutdown_connections 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f34400a3c30 0x7f34400a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3458106f70 0x7f34581a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3458101c30 0x7f345819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 --2- 172.21.15.12:0/2988534820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34581045a0 0x7f345819ff80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:59.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.447+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 >> 172.21.15.12:0/2988534820 conn(0x7f34580fd850 msgr2=0x7f34581025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:59.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.448+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 shutdown_connections 2024-04-26T21:54:59.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:54:59.448+0000 7f345ee00700 1 -- 172.21.15.12:0/2988534820 wait complete. 2024-04-26T21:55:00.257 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:55:00.261 INFO:tasks.cephadm:Deploying osd.4 on smithi077 with /dev/vg_nvme/lv_3... 2024-04-26T21:55:00.261 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:55:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:55:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:55:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:55:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:00.503 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:55:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:01 smithi077 ceph-mon[32280]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:55:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:01 smithi077 ceph-mon[32280]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:55:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:01 smithi012 ceph-mon[28184]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:55:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:01 smithi012 ceph-mon[28184]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:55:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:01 smithi154 ceph-mon[32578]: from='osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:55:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:01 smithi154 ceph-mon[32578]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:55:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:55:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:55:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] boot 2024-04-26T21:55:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:55:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:02.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:02 smithi012 ceph-mon[28184]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:55:02.421 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:55:02.422 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:55:02.422 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T21:55:02.422 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T21:55:02.422 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114589 s, 915 MB/s 2024-04-26T21:55:02.422 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T21:55:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:55:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:55:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] boot 2024-04-26T21:55:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:55:02.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:02.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:02 smithi077 ceph-mon[32280]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: osd.3 [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] boot 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:55:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:02 smithi154 ceph-mon[32578]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:55:03.053 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-04-26T21:55:03.290 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:55:03.671 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:03 smithi077 ceph-mon[32280]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:55:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:03 smithi012 ceph-mon[28184]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:55:03.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:03 smithi154 ceph-mon[32578]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:55:04.651 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:04 smithi077 ceph-mon[32280]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:04.651 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:04 smithi077 ceph-mon[32280]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- 172.21.15.77:0/3916189131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 msgr2=0x7f3c0c10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 --2- 172.21.15.77:0/3916189131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c10ba60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400b0a0 tx=0x7f3c0400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- 172.21.15.77:0/3916189131 shutdown_connections 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 --2- 172.21.15.77:0/3916189131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c10ba60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 --2- 172.21.15.77:0/3916189131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c104a70 0x7f3c0c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 --2- 172.21.15.77:0/3916189131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c1014f0 0x7f3c0c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- 172.21.15.77:0/3916189131 >> 172.21.15.77:0/3916189131 conn(0x7f3c0c0fd090 msgr2=0x7f3c0c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- 172.21.15.77:0/3916189131 shutdown_connections 2024-04-26T21:55:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- 172.21.15.77:0/3916189131 wait complete. 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 Processor -- start 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.680+0000 7f3c1292c700 1 -- start start 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c1014f0 0x7f3c0c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c104a70 0x7f3c0c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10f000 con 0x7f3c0c1014f0 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10ee80 con 0x7f3c0c101e90 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c1292c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10f180 con 0x7f3c0c104a70 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35406/0 (socket says 172.21.15.77:35406) 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 -- 172.21.15.77:0/1753860494 learned_addr learned my addr 172.21.15.77:0/1753860494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c10ec9700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c104a70 0x7f3c0c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 -- 172.21.15.77:0/1753860494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c104a70 msgr2=0x7f3c0c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0bfff700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c1014f0 0x7f3c0c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c104a70 0x7f3c0c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 -- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c1014f0 msgr2=0x7f3c0c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c1014f0 0x7f3c0c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 -- 172.21.15.77:0/1753860494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c1a1270 con 0x7f3c0c101e90 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c0b7fe700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d950 tx=0x7f3c0000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.681+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00016070 con 0x7f3c0c101e90 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.682+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c1a1560 con 0x7f3c0c101e90 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.682+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00009df0 con 0x7f3c0c101e90 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.682+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000f620 con 0x7f3c0c101e90 2024-04-26T21:55:04.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.682+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1a1b70 con 0x7f3c0c101e90 2024-04-26T21:55:04.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.682+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c06a110 con 0x7f3c0c101e90 2024-04-26T21:55:04.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.683+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c00019070 con 0x7f3c0c101e90 2024-04-26T21:55:04.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.684+0000 7f3c097fa700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:04.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.684+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f3c000ced80 con 0x7f3c0c101e90 2024-04-26T21:55:04.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.685+0000 7f3c0bfff700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:04.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.685+0000 7f3c0bfff700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 0x7f3bf40a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:04.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.689+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3c0009c980 con 0x7f3c0c101e90 2024-04-26T21:55:04.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:04 smithi012 ceph-mon[28184]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:04.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:04 smithi012 ceph-mon[28184]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:55:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:04 smithi154 ceph-mon[32578]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:04 smithi154 ceph-mon[32578]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:55:04.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:04.898+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3c0c063920 con 0x7f3bf40a3c40 2024-04-26T21:55:05.795 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:05 smithi077 ceph-mon[32280]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:55:05.795 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:05.796 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:05.796 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:05 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:05.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:05 smithi012 ceph-mon[28184]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:55:05.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:05.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:05.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:05 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:05 smithi154 ceph-mon[32578]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:55:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:05.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:05 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:06.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:06.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:06.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:06.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:06.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:07 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi012 2024-04-26T21:55:07.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:07 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 8547M 2024-04-26T21:55:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:07 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi012 2024-04-26T21:55:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:07 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 8547M 2024-04-26T21:55:07.962 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:07 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi012 2024-04-26T21:55:07.962 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:07 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 8547M 2024-04-26T21:55:08.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/1594937282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]': finished 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:08.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:08 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/2944254695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/1594937282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]': finished 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:08 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/2944254695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/1594937282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]: dispatch 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13cc095e-3560-47a1-bca1-4dc3cbbc8198"}]': finished 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:08.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:08 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/2944254695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:10.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:10 smithi012 ceph-mon[28184]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:55:10.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:10 smithi154 ceph-mon[32578]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:55:10.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:10 smithi077 ceph-mon[32280]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:55:12.819 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:12 smithi077 ceph-mon[32280]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:55:12.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:12 smithi012 ceph-mon[28184]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:55:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:12 smithi154 ceph-mon[32578]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:55:14.611 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:14 smithi077 ceph-mon[32280]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:55:14.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:14 smithi012 ceph-mon[28184]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:55:14.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:14 smithi154 ceph-mon[32578]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:55:15.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:55:15.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:15.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:15 smithi077 ceph-mon[32280]: Deploying daemon osd.4 on smithi077 2024-04-26T21:55:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:55:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:15 smithi012 ceph-mon[28184]: Deploying daemon osd.4 on smithi077 2024-04-26T21:55:15.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:55:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:15 smithi154 ceph-mon[32578]: Deploying daemon osd.4 on smithi077 2024-04-26T21:55:16.613 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:16 smithi077 ceph-mon[32280]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:16.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:16 smithi012 ceph-mon[28184]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:16.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:16 smithi154 ceph-mon[32578]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:18.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:18 smithi012 ceph-mon[28184]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:18.862 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:18 smithi077 ceph-mon[32280]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:18.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:18 smithi154 ceph-mon[32578]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:20 smithi012 ceph-mon[28184]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:20 smithi154 ceph-mon[32578]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:20.898 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:20 smithi077 ceph-mon[32280]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:20.898 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:20.898 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:20.898 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:22 smithi077 ceph-mon[32280]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:22.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:22 smithi012 ceph-mon[28184]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:22.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:22 smithi154 ceph-mon[32578]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:24.029 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 4 on host 'smithi077' 2024-04-26T21:55:24.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.029+0000 7f3c097fa700 1 -- 172.21.15.77:0/1753860494 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c0c063920 con 0x7f3bf40a3c40 2024-04-26T21:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 msgr2=0x7f3bf40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 0x7f3bf40a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc008040 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 msgr2=0x7f3c0c199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d950 tx=0x7f3c0000dc60 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 shutdown_connections 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c0c104a70 0x7f3c0c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c0c1014f0 0x7f3c0c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 --2- 172.21.15.77:0/1753860494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c0c101e90 0x7f3c0c199b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 >> 172.21.15.77:0/1753860494 conn(0x7f3c0c0fd090 msgr2=0x7f3c0c107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 shutdown_connections 2024-04-26T21:55:24.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:55:24.031+0000 7f3c1292c700 1 -- 172.21.15.77:0/1753860494 wait complete. 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.668 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:24 smithi077 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.695 DEBUG:teuthology.orchestra.run.smithi077:osd.4> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:55:24.698 INFO:tasks.cephadm:Deploying osd.5 on smithi154 with /dev/vg_nvme/lv_3... 2024-04-26T21:55:24.698 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:55:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:24 smithi012 ceph-mon[28184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:24 smithi154 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:55:24.944 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:26 smithi154 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:26.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:55:26.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:55:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.483 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T21:55:26.311+0000 7f9268443700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:55:26.483 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 21:55:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T21:55:26.319+0000 7f9264236700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:55:26.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:26.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:55:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: from='osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:55:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:26.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:26 smithi012 ceph-mon[28184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:55:26.997 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:55:26.997 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:55:26.998 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-26T21:55:26.998 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-26T21:55:26.998 INFO:teuthology.orchestra.run.smithi154.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111703 s, 939 MB/s 2024-04-26T21:55:26.998 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-26T21:55:27.627 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:27 smithi077 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:55:27.627 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:27 smithi077 ceph-mon[32280]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:55:27.627 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.627 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:27 smithi154 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:55:27.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:27 smithi154 ceph-mon[32578]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:55:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.713 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-04-26T21:55:27.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:27 smithi012 ceph-mon[28184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:55:27.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:27 smithi012 ceph-mon[28184]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:55:27.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:27.962 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:55:28.339 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:55:28.339 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:55:28.339 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:28.339 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] boot 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi077 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.340 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.341 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.341 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.341 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] boot 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi077 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: osd.4 [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] boot 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi077 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 8544M 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:29.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.509+0000 7fc01702a700 1 -- 172.21.15.154:0/1110621635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010105020 msgr2=0x7fc01010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.509+0000 7fc01702a700 1 --2- 172.21.15.154:0/1110621635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010105020 0x7fc01010b8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc00800b0a0 tx=0x7fc00800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- 172.21.15.154:0/1110621635 shutdown_connections 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- 172.21.15.154:0/1110621635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010105020 0x7fc01010b8d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- 172.21.15.154:0/1110621635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 0x7fc0101038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- 172.21.15.154:0/1110621635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc010104680 0x7fc010104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- 172.21.15.154:0/1110621635 >> 172.21.15.154:0/1110621635 conn(0x7fc0100fea20 msgr2=0x7fc010100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- 172.21.15.154:0/1110621635 shutdown_connections 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- 172.21.15.154:0/1110621635 wait complete. 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 Processor -- start 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- start start 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 0x7fc01019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc010105020 0x7fc0101a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010ef70 con 0x7fc010105020 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010edf0 con 0x7fc010103480 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.510+0000 7fc01702a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010f0f0 con 0x7fc010104680 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59580/0 (socket says 172.21.15.154:59580) 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 -- 172.21.15.154:0/2328159241 learned_addr learned my addr 172.21.15.154:0/2328159241 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc014dc6700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 0x7fc01019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 -- 172.21.15.154:0/2328159241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 msgr2=0x7fc01019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 0x7fc01019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 -- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc010105020 msgr2=0x7fc0101a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc010105020 0x7fc0101a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 -- 172.21.15.154:0/2328159241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101a3420 con 0x7fc010104680 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00ffff700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b560 tx=0x7fbffc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc015020 con 0x7fc010104680 2024-04-26T21:55:29.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101a3710 con 0x7fc010104680 2024-04-26T21:55:29.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc011ce0 con 0x7fc010104680 2024-04-26T21:55:29.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc010d20 con 0x7fc010104680 2024-04-26T21:55:29.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.511+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101a3d20 con 0x7fc010104680 2024-04-26T21:55:29.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.512+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01006a110 con 0x7fc010104680 2024-04-26T21:55:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.513+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbffc018020 con 0x7fc010104680 2024-04-26T21:55:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.514+0000 7fc00dffb700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 0x7fc0000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.514+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fbffc0cea70 con 0x7fc010104680 2024-04-26T21:55:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.514+0000 7fc014dc6700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 0x7fc0000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.515+0000 7fc014dc6700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 0x7fc0000a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0101044e0 tx=0x7fc00400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.519+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbffc01f020 con 0x7fc010104680 2024-04-26T21:55:29.519 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:29 smithi154 ceph-mon[32578]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:55:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:29.723+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc010076d70 con 0x7fc0000a3cc0 2024-04-26T21:55:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:29 smithi077 ceph-mon[32280]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:55:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:29 smithi012 ceph-mon[28184]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:55:30.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:30 smithi154 ceph-mon[32578]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:30.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:30 smithi154 ceph-mon[32578]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:30.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:30 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:30.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:30 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:30.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:30 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:30.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:30 smithi077 ceph-mon[32280]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:30.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:30 smithi077 ceph-mon[32280]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:30.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:30 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:30.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:30 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:30.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:30 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:30 smithi012 ceph-mon[28184]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:30 smithi012 ceph-mon[28184]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:30 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/3462128828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]': finished 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:55:32.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:32.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:32.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/3462128828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]': finished 2024-04-26T21:55:32.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:55:32.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:32.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:32.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/3462128828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]: dispatch 2024-04-26T21:55:32.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38bc196f-d165-47ad-815e-da841218d3a2"}]': finished 2024-04-26T21:55:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:55:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:33.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:33 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/1812687881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:33 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/1812687881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:33 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/1812687881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:55:34.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:34 smithi154 ceph-mon[32578]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:34.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:34 smithi077 ceph-mon[32280]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:34.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:34 smithi012 ceph-mon[28184]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:36.524 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:36 smithi154 ceph-mon[32578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:36.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:36 smithi077 ceph-mon[32280]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:36.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:36 smithi012 ceph-mon[28184]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:38.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:38 smithi154 ceph-mon[32578]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:38.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:38 smithi077 ceph-mon[32280]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:38.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:38 smithi012 ceph-mon[28184]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:40 smithi154 ceph-mon[32578]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:55:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:40 smithi154 ceph-mon[32578]: Deploying daemon osd.5 on smithi154 2024-04-26T21:55:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:40 smithi077 ceph-mon[32280]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:55:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:40 smithi077 ceph-mon[32280]: Deploying daemon osd.5 on smithi154 2024-04-26T21:55:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:40 smithi012 ceph-mon[28184]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:55:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:40 smithi012 ceph-mon[28184]: Deploying daemon osd.5 on smithi154 2024-04-26T21:55:42.621 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:42 smithi154 ceph-mon[32578]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:42.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:42 smithi077 ceph-mon[32280]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:42.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:42 smithi012 ceph-mon[28184]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:44 smithi154 ceph-mon[32578]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:44 smithi077 ceph-mon[32280]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:44 smithi012 ceph-mon[28184]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:46 smithi012 ceph-mon[28184]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:46 smithi154 ceph-mon[32578]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:46.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:46.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:46 smithi077 ceph-mon[32280]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:48.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.716+0000 7fc00dffb700 1 -- 172.21.15.154:0/2328159241 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc010076d70 con 0x7fc0000a3cc0 2024-04-26T21:55:48.717 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 5 on host 'smithi154' 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 msgr2=0x7fc0000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 0x7fc0000a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0101044e0 tx=0x7fc00400a380 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 msgr2=0x7fc01019bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b560 tx=0x7fbffc00b870 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 shutdown_connections 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc010105020 0x7fc0101a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc010103480 0x7fc01019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0000a3cc0 0x7fc0000a6180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 --2- 172.21.15.154:0/2328159241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc010104680 0x7fc01019bd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.718+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 >> 172.21.15.154:0/2328159241 conn(0x7fc0100fea20 msgr2=0x7fc010075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.719+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 shutdown_connections 2024-04-26T21:55:48.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:55:48.719+0000 7fc01702a700 1 -- 172.21.15.154:0/2328159241 wait complete. 2024-04-26T21:55:49.020 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:49.020 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.020 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.020 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:49.020 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:49.021 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.021 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.021 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.021 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:48 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:48 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:49.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:49.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:48 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:49.356 DEBUG:teuthology.orchestra.run.smithi154:osd.5> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:55:49.358 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-04-26T21:55:49.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:55:49.839 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:55:49.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:49 smithi154 ceph-mon[32578]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:49.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:49 smithi154 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:50.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:49 smithi012 ceph-mon[28184]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:50.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:49 smithi012 ceph-mon[28184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:50.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:49 smithi077 ceph-mon[32280]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:50.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:49 smithi077 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:50 smithi154 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:50 smithi012 ceph-mon[28184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:55:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:55:51.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: from='osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:51.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:50 smithi077 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:55:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:55:51.851 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T21:55:51.594+0000 7f19aac86700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:55:51.851 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T21:55:51.602+0000 7f19a2270700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: from='osd.5 ' entity='osd.5' 2024-04-26T21:55:51.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: from='osd.5 ' entity='osd.5' 2024-04-26T21:55:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:55:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:55:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:55:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: from='osd.5 ' entity='osd.5' 2024-04-26T21:55:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:53.021 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] boot 2024-04-26T21:55:53.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:55:53.023 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324584 s, 323 MB/s 2024-04-26T21:55:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] boot 2024-04-26T21:55:53.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:55:53.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:55:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:55:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:55:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: osd.5 [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] boot 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:55:53.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:55:53.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-04-26T21:55:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:53 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi154 2024-04-26T21:55:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:53 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-26T21:55:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:53 smithi012 ceph-mon[28184]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:55:54.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:53 smithi012 ceph-mon[28184]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:55:54.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:53 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi154 2024-04-26T21:55:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:53 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-26T21:55:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:53 smithi154 ceph-mon[32578]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:55:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:53 smithi154 ceph-mon[32578]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:55:54.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:53 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi154 2024-04-26T21:55:54.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:53 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-26T21:55:54.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:53 smithi077 ceph-mon[32280]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:55:54.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:53 smithi077 ceph-mon[32280]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:55:54.302 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:55:55.716 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:55 smithi012 ceph-mon[28184]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:55:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:55 smithi154 ceph-mon[32578]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:55:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:55 smithi077 ceph-mon[32280]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:55:56.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:56 smithi012 ceph-mon[28184]: pgmap v157: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:55:56.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:56 smithi012 ceph-mon[28184]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:55:56.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:56 smithi012 ceph-mon[28184]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:55:56.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.575+0000 7f46e0c00700 1 -- 172.21.15.12:0/3026196197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc1090a0 msgr2=0x7f46dc10b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:56.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.575+0000 7f46e0c00700 1 --2- 172.21.15.12:0/3026196197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc1090a0 0x7f46dc10b510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f46d400b0a0 tx=0x7f46d400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 -- 172.21.15.12:0/3026196197 shutdown_connections 2024-04-26T21:55:56.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 --2- 172.21.15.12:0/3026196197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc1090a0 0x7f46dc10b510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 --2- 172.21.15.12:0/3026196197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc103e40 0x7f46dc106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 --2- 172.21.15.12:0/3026196197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 0x7f46dc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 -- 172.21.15.12:0/3026196197 >> 172.21.15.12:0/3026196197 conn(0x7f46dc0fd830 msgr2=0x7f46dc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 -- 172.21.15.12:0/3026196197 shutdown_connections 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.576+0000 7f46e0c00700 1 -- 172.21.15.12:0/3026196197 wait complete. 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.577+0000 7f46e0c00700 1 Processor -- start 2024-04-26T21:55:56.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.577+0000 7f46e0c00700 1 -- start start 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 0x7f46dc101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc1090a0 0x7f46dc102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc10ee70 con 0x7f46dc1090a0 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc10ecf0 con 0x7f46dc103e40 2024-04-26T21:55:56.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46e0c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc10eff0 con 0x7f46dc106770 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46db400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46dbe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc1090a0 0x7f46dc102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46db400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60632/0 (socket says 172.21.15.12:60632) 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46db400700 1 -- 172.21.15.12:0/2941803690 learned_addr learned my addr 172.21.15.12:0/2941803690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.578+0000 7f46daa00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 0x7f46dc101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 -- 172.21.15.12:0/2941803690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 msgr2=0x7f46dc101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 0x7f46dc101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 -- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc1090a0 msgr2=0x7f46dc102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc1090a0 0x7f46dc102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 -- 172.21.15.12:0/2941803690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc102c00 con 0x7f46dc103e40 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46db400700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00e9a0 tx=0x7f46cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:56.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00ca10 con 0x7f46dc103e40 2024-04-26T21:55:56.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc00cbb0 con 0x7f46dc103e40 2024-04-26T21:55:56.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46dc1a35e0 con 0x7f46dc103e40 2024-04-26T21:55:56.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc018680 con 0x7f46dc103e40 2024-04-26T21:55:56.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.579+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46dc1a3b40 con 0x7f46dc103e40 2024-04-26T21:55:56.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.581+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46dc06a110 con 0x7f46dc103e40 2024-04-26T21:55:56.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.582+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46cc010430 con 0x7f46dc103e40 2024-04-26T21:55:56.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.584+0000 7f46d3e00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 0x7f46c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:56.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.584+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f46cc0cf730 con 0x7f46dc103e40 2024-04-26T21:55:56.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.584+0000 7f46daa00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 0x7f46c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:56.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.585+0000 7f46daa00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 0x7f46c00a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009680 tx=0x7f46c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:56.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.589+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f46cc09cf30 con 0x7f46dc103e40 2024-04-26T21:55:56.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:56 smithi154 ceph-mon[32578]: pgmap v157: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:55:56.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:56 smithi154 ceph-mon[32578]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:55:56.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:56 smithi154 ceph-mon[32578]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:55:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:55:56.829+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f46dc02e870 con 0x7f46c00a3cc0 2024-04-26T21:55:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:56 smithi077 ceph-mon[32280]: pgmap v157: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:55:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:56 smithi077 ceph-mon[32280]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T21:55:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:56 smithi077 ceph-mon[32280]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:55:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:57 smithi012 ceph-mon[28184]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:55:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:57 smithi012 ceph-mon[28184]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:57.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:57 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:57 smithi154 ceph-mon[32578]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:55:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:57 smithi154 ceph-mon[32578]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:57 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:57 smithi077 ceph-mon[32280]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:55:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:57 smithi077 ceph-mon[32280]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:55:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:55:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:57 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:58.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:55:58 smithi012 ceph-mon[28184]: pgmap v160: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:55:58.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:55:58 smithi154 ceph-mon[32578]: pgmap v160: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:55:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:55:58 smithi077 ceph-mon[32280]: pgmap v160: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:56:00.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:00 smithi012 ceph-mon[28184]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:56:00.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:00 smithi154 ceph-mon[32578]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:56:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:00 smithi077 ceph-mon[32280]: pgmap v161: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T21:56:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:01 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]: dispatch 2024-04-26T21:56:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:01 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]': finished 2024-04-26T21:56:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:01 smithi012 ceph-mon[28184]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:56:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:01 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:01 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]: dispatch 2024-04-26T21:56:01.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:01 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]': finished 2024-04-26T21:56:01.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:01 smithi154 ceph-mon[32578]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:56:01.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:01 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:01 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]: dispatch 2024-04-26T21:56:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:01 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2660433934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5475e1d-8f75-4b95-b8d5-45916e0ed62f"}]': finished 2024-04-26T21:56:01.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:01 smithi077 ceph-mon[32280]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:56:01.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:01 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:02 smithi012 ceph-mon[28184]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T21:56:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:02 smithi012 ceph-mon[28184]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:56:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:02 smithi012 ceph-mon[28184]: Cluster is now healthy 2024-04-26T21:56:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:02 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2503029326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:02 smithi154 ceph-mon[32578]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T21:56:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:02 smithi154 ceph-mon[32578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:56:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:02 smithi154 ceph-mon[32578]: Cluster is now healthy 2024-04-26T21:56:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:02 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2503029326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:02 smithi077 ceph-mon[32280]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T21:56:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:02 smithi077 ceph-mon[32280]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T21:56:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:02 smithi077 ceph-mon[32280]: Cluster is now healthy 2024-04-26T21:56:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:02 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2503029326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:04.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:04 smithi012 ceph-mon[28184]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:56:04.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:04 smithi154 ceph-mon[32578]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:56:04.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:04 smithi077 ceph-mon[32280]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:56:06.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:06 smithi012 ceph-mon[28184]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:56:06.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:06 smithi154 ceph-mon[32578]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:56:06.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:06 smithi077 ceph-mon[32280]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:56:08.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:08 smithi012 ceph-mon[28184]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:08 smithi154 ceph-mon[32578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:08 smithi077 ceph-mon[32280]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:10.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:10 smithi012 ceph-mon[28184]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:10.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:10 smithi154 ceph-mon[32578]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:10.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:10 smithi077 ceph-mon[32280]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:56:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:11 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:56:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:11 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:11.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:11 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:56:11.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:11 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:11 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:56:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:11 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:12.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:12 smithi012 ceph-mon[28184]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:12.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:12 smithi012 ceph-mon[28184]: Deploying daemon osd.6 on smithi012 2024-04-26T21:56:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:12 smithi154 ceph-mon[32578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:12 smithi154 ceph-mon[32578]: Deploying daemon osd.6 on smithi012 2024-04-26T21:56:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:12 smithi077 ceph-mon[32280]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:12 smithi077 ceph-mon[32280]: Deploying daemon osd.6 on smithi012 2024-04-26T21:56:14.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:14 smithi012 ceph-mon[28184]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:14.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:14 smithi154 ceph-mon[32578]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:14 smithi077 ceph-mon[32280]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:16.746 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:16 smithi012 ceph-mon[28184]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:16 smithi154 ceph-mon[32578]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:16.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:16 smithi077 ceph-mon[32280]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:18.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:18 smithi012 ceph-mon[28184]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:18.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:18 smithi154 ceph-mon[32578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:18 smithi077 ceph-mon[32280]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:20 smithi012 ceph-mon[28184]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:20 smithi154 ceph-mon[32578]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:20.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:20 smithi077 ceph-mon[32280]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:20.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:20.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:22.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:22 smithi012 ceph-mon[28184]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:22.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:22 smithi154 ceph-mon[32578]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:22.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:22 smithi077 ceph-mon[32280]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:24 smithi154 ceph-mon[32578]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:24 smithi154 ceph-mon[32578]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:24 smithi154 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:24 smithi077 ceph-mon[32280]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:24 smithi077 ceph-mon[32280]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:24 smithi077 ceph-mon[32280]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:24 smithi012 ceph-mon[28184]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:24 smithi012 ceph-mon[28184]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:24 smithi012 ceph-mon[28184]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:56:25.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.285+0000 7f46d3e00700 1 -- 172.21.15.12:0/2941803690 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46dc02e870 con 0x7f46c00a3cc0 2024-04-26T21:56:25.287 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-04-26T21:56:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 msgr2=0x7f46c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 0x7f46c00a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009680 tx=0x7f46c8009260 comp rx=0 tx=0).stop 2024-04-26T21:56:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 msgr2=0x7f46dc103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00e9a0 tx=0x7f46cc00ecb0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 shutdown_connections 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46dc1090a0 0x7f46dc102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46dc103e40 0x7f46dc103750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f46c00a3cc0 0x7f46c00a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 --2- 172.21.15.12:0/2941803690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46dc106770 0x7f46dc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 >> 172.21.15.12:0/2941803690 conn(0x7f46dc0fd830 msgr2=0x7f46dc104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 shutdown_connections 2024-04-26T21:56:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:56:25.289+0000 7f46e0c00700 1 -- 172.21.15.12:0/2941803690 wait complete. 2024-04-26T21:56:26.192 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:56:26.197 INFO:tasks.cephadm:Deploying osd.7 on smithi077 with /dev/vg_nvme/lv_2... 2024-04-26T21:56:26.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:56:26.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:56:26.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:56:26.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:26.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:26.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:26.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:26.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:26.441 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:56:26.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:26.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:26.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:27.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:27 smithi012 ceph-mon[28184]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:56:27.186 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:27 smithi012 ceph-mon[28184]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:56:27.187 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:27.187 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:27.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:27 smithi077 ceph-mon[32280]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:56:27.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:27 smithi077 ceph-mon[32280]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:56:27.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:27.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:27.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:27 smithi154 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:56:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:27 smithi154 ceph-mon[32578]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:56:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] boot 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:28.179 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:28 smithi012 ceph-mon[28184]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] boot 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:28 smithi077 ceph-mon[32280]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:28.388 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:56:28.388 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:56:28.388 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T21:56:28.388 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T21:56:28.388 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113535 s, 924 MB/s 2024-04-26T21:56:28.389 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T21:56:28.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:56:28.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:56:28.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: osd.6 [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] boot 2024-04-26T21:56:28.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:56:28.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:56:28.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:28 smithi154 ceph-mon[32578]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:56:29.032 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-04-26T21:56:29.267 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:56:29.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:29 smithi012 ceph-mon[28184]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:56:29.388 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:29 smithi077 ceph-mon[32280]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:56:29.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:29 smithi154 ceph-mon[32578]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:56:30.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:30 smithi012 ceph-mon[28184]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:30.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:30 smithi154 ceph-mon[32578]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:30.405 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:30 smithi077 ceph-mon[32280]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- 172.21.15.77:0/2290484282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0102390 msgr2=0x7fd5b01027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2290484282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0102390 0x7fd5b01027b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0099c0 tx=0x7fd5ac009cd0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- 172.21.15.77:0/2290484282 shutdown_connections 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2290484282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103f30 0x7fd5b010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2290484282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0102390 0x7fd5b01027b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2290484282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b0103590 0x7fd5b01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- 172.21.15.77:0/2290484282 >> 172.21.15.77:0/2290484282 conn(0x7fd5b00fd950 msgr2=0x7fd5b00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- 172.21.15.77:0/2290484282 shutdown_connections 2024-04-26T21:56:30.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- 172.21.15.77:0/2290484282 wait complete. 2024-04-26T21:56:30.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 Processor -- start 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.904+0000 7fd5b7906700 1 -- start start 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b0102390 0x7fd5b01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103590 0x7fd5b0199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b010ded0 con 0x7fd5b0102390 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b010dd50 con 0x7fd5b0103f30 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b010e050 con 0x7fd5b0103590 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41264/0 (socket says 172.21.15.77:41264) 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 -- 172.21.15.77:0/2850174925 learned_addr learned my addr 172.21.15.77:0/2850174925 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b4ea1700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103590 0x7fd5b0199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 -- 172.21.15.77:0/2850174925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103590 msgr2=0x7fd5b0199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103590 0x7fd5b0199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 -- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b0102390 msgr2=0x7fd5b01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b0102390 0x7fd5b01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 -- 172.21.15.77:0/2850174925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01a1330 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b5ea3700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8007ee0 tx=0x7fd5a800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800ddd0 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8004020 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01a1620 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8015660 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.905+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01a1bb0 con 0x7fd5b0103f30 2024-04-26T21:56:30.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.906+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b006a110 con 0x7fd5b0103f30 2024-04-26T21:56:30.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.908+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5a80041d0 con 0x7fd5b0103f30 2024-04-26T21:56:30.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.909+0000 7fd5a67fc700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 0x7fd59c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.909+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fd5a80cf950 con 0x7fd5b0103f30 2024-04-26T21:56:30.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.909+0000 7fd5b56a2700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 0x7fd59c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.910+0000 7fd5b56a2700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 0x7fd59c0a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098e0 tx=0x7fd5ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:30.912+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd5a8010070 con 0x7fd5b0103f30 2024-04-26T21:56:31.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:31.122+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd5b00fe730 con 0x7fd59c0a3cc0 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi012 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 5698M 2024-04-26T21:56:32.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:32 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi012 2024-04-26T21:56:32.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:32.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 5698M 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:32.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.393 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:32.393 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.393 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.393 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:32 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi012 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 5698M 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.415 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:32.416 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:32 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/483722479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]': finished 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:56:34.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:34 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/483722479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]': finished 2024-04-26T21:56:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:56:34.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:34 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/483722479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]: dispatch 2024-04-26T21:56:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00d199a4-d87f-4184-b154-13d6f3dd67e9"}]': finished 2024-04-26T21:56:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:56:34.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:35.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:35 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/670524630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:35.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:35 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/670524630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:35 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/670524630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:56:36.383 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:36 smithi077 ceph-mon[32280]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:36.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:36 smithi154 ceph-mon[32578]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:36.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:36 smithi012 ceph-mon[28184]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:38.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:38 smithi154 ceph-mon[32578]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:38.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:38 smithi077 ceph-mon[32280]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:38.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:38 smithi012 ceph-mon[28184]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:40.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:40 smithi077 ceph-mon[32280]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:40.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:40 smithi154 ceph-mon[32578]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:40 smithi012 ceph-mon[28184]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:41.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:56:41.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:41.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:56:41.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:41.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:56:41.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:42.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:42 smithi077 ceph-mon[32280]: Deploying daemon osd.7 on smithi077 2024-04-26T21:56:42.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:42 smithi077 ceph-mon[32280]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:42.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:42 smithi154 ceph-mon[32578]: Deploying daemon osd.7 on smithi077 2024-04-26T21:56:42.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:42 smithi154 ceph-mon[32578]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:42.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:42 smithi012 ceph-mon[28184]: Deploying daemon osd.7 on smithi077 2024-04-26T21:56:42.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:42 smithi012 ceph-mon[28184]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:44.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:44 smithi154 ceph-mon[32578]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:44.436 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:44 smithi077 ceph-mon[32280]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:44.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:44 smithi012 ceph-mon[28184]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:46.322 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:46 smithi077 ceph-mon[32280]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:46.376 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:46 smithi012 ceph-mon[28184]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:46.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:46 smithi154 ceph-mon[32578]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:47.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:47.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:47.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:48 smithi077 ceph-mon[32280]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:49.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:48 smithi012 ceph-mon[28184]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:49.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:48 smithi154 ceph-mon[32578]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:50.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.559+0000 7fd5a67fc700 1 -- 172.21.15.77:0/2850174925 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5b00fe730 con 0x7fd59c0a3cc0 2024-04-26T21:56:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 7 on host 'smithi077' 2024-04-26T21:56:50.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 msgr2=0x7fd59c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 0x7fd59c0a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098e0 tx=0x7fd5ac000bc0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 msgr2=0x7fd5b01a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8007ee0 tx=0x7fd5a800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 shutdown_connections 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5b0103f30 0x7fd5b01a0c90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b0102390 0x7fd5b01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd59c0a3cc0 0x7fd59c0a6180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 --2- 172.21.15.77:0/2850174925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5b0103590 0x7fd5b0199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 >> 172.21.15.77:0/2850174925 conn(0x7fd5b00fd950 msgr2=0x7fd5b00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 shutdown_connections 2024-04-26T21:56:50.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:56:50.561+0000 7fd5b7906700 1 -- 172.21.15.77:0/2850174925 wait complete. 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:50.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:50.735 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.735 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.735 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:50.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:50.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:51.157 DEBUG:teuthology.orchestra.run.smithi077:osd.7> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:56:51.160 INFO:tasks.cephadm:Deploying osd.8 on smithi154 with /dev/vg_nvme/lv_2... 2024-04-26T21:56:51.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:56:51.405 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:56:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:51 smithi077 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:56:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:51 smithi077 ceph-mon[32280]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:51 smithi077 ceph-mon[32280]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:56:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:51 smithi077 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:51 smithi154 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:56:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:51 smithi154 ceph-mon[32578]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:51 smithi154 ceph-mon[32578]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:56:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:51.851 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:51 smithi154 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:51 smithi012 ceph-mon[28184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:56:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:51 smithi012 ceph-mon[28184]: from='osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:51 smithi012 ceph-mon[28184]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:56:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:52.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:51 smithi012 ceph-mon[28184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:56:52.618 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T21:56:52.334+0000 7fa9148a0700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:56:52.619 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T21:56:52.345+0000 7fa90c68b700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: from='osd.7 ' entity='osd.7' 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] boot 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:56:52.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:52.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: from='osd.7 ' entity='osd.7' 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] boot 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:56:52.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:53.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:56:53.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:56:53.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:56:53.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:53.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: from='osd.7 ' entity='osd.7' 2024-04-26T21:56:53.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: osd.7 [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] boot 2024-04-26T21:56:53.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:56:53.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114661 s, 914 MB/s 2024-04-26T21:56:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-26T21:56:53.871 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:53 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:56:53.871 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:53 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:56:53.872 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:53 smithi154 ceph-mon[32578]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:56:53.900 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-04-26T21:56:53.947 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:53 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:56:53.947 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:53 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:56:53.947 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:53 smithi077 ceph-mon[32280]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:56:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:53 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:56:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:53 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:56:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:53 smithi012 ceph-mon[28184]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:56:54.167 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:54.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:54.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:54 smithi154 ceph-mon[32578]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:56:54.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:54.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:54.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:54 smithi077 ceph-mon[32280]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: pgmap v197: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:55.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:55.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:55.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:56:55.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:54 smithi012 ceph-mon[28184]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 -- 172.21.15.154:0/1881630936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 msgr2=0x7f35a010a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 --2- 172.21.15.154:0/1881630936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a010a7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f359800b0a0 tx=0x7f359800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 -- 172.21.15.154:0/1881630936 shutdown_connections 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 --2- 172.21.15.154:0/1881630936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a010a7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 --2- 172.21.15.154:0/1881630936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35a0102370 0x7f35a0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 --2- 172.21.15.154:0/1881630936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0103570 0x7f35a01039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.712+0000 7f35a869c700 1 -- 172.21.15.154:0/1881630936 >> 172.21.15.154:0/1881630936 conn(0x7f35a00fd910 msgr2=0x7f35a00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- 172.21.15.154:0/1881630936 shutdown_connections 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- 172.21.15.154:0/1881630936 wait complete. 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 Processor -- start 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- start start 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 0x7f35a01062b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35a0103570 0x7f35a0104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010de90 con 0x7f35a0103570 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010dd10 con 0x7f35a0102370 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a869c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010e010 con 0x7f35a0103f10 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a6c39700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a6c39700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55638/0 (socket says 172.21.15.154:55638) 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.713+0000 7f35a6c39700 1 -- 172.21.15.154:0/675044549 learned_addr learned my addr 172.21.15.154:0/675044549 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6438700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 0x7f35a01062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 -- 172.21.15.154:0/675044549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 msgr2=0x7f35a01062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 0x7f35a01062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 -- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35a0103570 msgr2=0x7f35a0104950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35a0103570 0x7f35a0104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 -- 172.21.15.154:0/675044549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a0105760 con 0x7f35a0103f10 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6438700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 0x7f35a01062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a6c39700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f359800b070 tx=0x7f3598004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598021030 con 0x7f35a0103f10 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a01a3730 con 0x7f35a0103f10 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3598007be0 con 0x7f35a0103f10 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598004790 con 0x7f35a0103f10 2024-04-26T21:56:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.714+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a01a3d10 con 0x7f35a0103f10 2024-04-26T21:56:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.715+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a006a110 con 0x7f35a0103f10 2024-04-26T21:56:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.716+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3598024070 con 0x7f35a0103f10 2024-04-26T21:56:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.717+0000 7f35977fe700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 0x7f358c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.717+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f359800f020 con 0x7f35a0103f10 2024-04-26T21:56:55.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.717+0000 7f35a6438700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 0x7f358c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:55.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.718+0000 7f35a6438700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 0x7f358c0a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35a01033d0 tx=0x7f359c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:55.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.721+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f359809dcb0 con 0x7f35a0103f10 2024-04-26T21:56:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:55 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi077 2024-04-26T21:56:55.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:55 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-26T21:56:55.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:56:55.929+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f35a00ffae0 con 0x7f358c0a3c40 2024-04-26T21:56:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:55 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi077 2024-04-26T21:56:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:55 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-26T21:56:56.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:55 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi077 2024-04-26T21:56:56.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:55 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 5696M 2024-04-26T21:56:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:56 smithi077 ceph-mon[32280]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:56 smithi077 ceph-mon[32280]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:56:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:56.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:56 smithi012 ceph-mon[28184]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:56 smithi012 ceph-mon[28184]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:56:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:57.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:56 smithi154 ceph-mon[32578]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:57.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:56 smithi154 ceph-mon[32578]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:56:57.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:56:57.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:56:57.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:57.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:57 smithi154 ceph-mon[32578]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:57.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:57 smithi154 ceph-mon[32578]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:56:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:57 smithi077 ceph-mon[32280]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:57 smithi077 ceph-mon[32280]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:56:58.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:57 smithi012 ceph-mon[28184]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:56:58.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:57 smithi012 ceph-mon[28184]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/1738875454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]': finished 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:56:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:56:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/1738875454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]': finished 2024-04-26T21:56:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:56:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/1738875454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]: dispatch 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03a5cda5-c502-4af5-88e3-3196bf23380d"}]': finished 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:56:59.078 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:00.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:56:59 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/2820818275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:00.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:56:59 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/2820818275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:00.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:56:59 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/2820818275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:01.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:00 smithi012 ceph-mon[28184]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:01.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:00 smithi012 ceph-mon[28184]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:57:01.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:00 smithi012 ceph-mon[28184]: Cluster is now healthy 2024-04-26T21:57:01.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:00 smithi154 ceph-mon[32578]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:00 smithi154 ceph-mon[32578]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:57:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:00 smithi154 ceph-mon[32578]: Cluster is now healthy 2024-04-26T21:57:01.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:00 smithi077 ceph-mon[32280]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:01.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:00 smithi077 ceph-mon[32280]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-26T21:57:01.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:00 smithi077 ceph-mon[32280]: Cluster is now healthy 2024-04-26T21:57:03.017 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:02 smithi154 ceph-mon[32578]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:02 smithi012 ceph-mon[28184]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:03.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:02 smithi077 ceph-mon[32280]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:04.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:04 smithi154 ceph-mon[32578]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:05.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:04 smithi012 ceph-mon[28184]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:04 smithi077 ceph-mon[32280]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:07.035 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:06 smithi077 ceph-mon[32280]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:07.035 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:57:07.035 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:06 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:06 smithi012 ceph-mon[28184]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:57:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:06 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:07.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:06 smithi154 ceph-mon[32578]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:07.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:57:07.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:06 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:07.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:07 smithi154 ceph-mon[32578]: Deploying daemon osd.8 on smithi154 2024-04-26T21:57:08.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:07 smithi012 ceph-mon[28184]: Deploying daemon osd.8 on smithi154 2024-04-26T21:57:08.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:07 smithi077 ceph-mon[32280]: Deploying daemon osd.8 on smithi154 2024-04-26T21:57:09.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:08 smithi012 ceph-mon[28184]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:09.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:08 smithi154 ceph-mon[32578]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:09.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:08 smithi077 ceph-mon[32280]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:11.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:10 smithi012 ceph-mon[28184]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:11.115 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:10 smithi154 ceph-mon[32578]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:11.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:10 smithi077 ceph-mon[32280]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:12 smithi154 ceph-mon[32578]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:12.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:12 smithi077 ceph-mon[32280]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:12.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:12.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:12.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:12 smithi012 ceph-mon[28184]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:14 smithi077 ceph-mon[32280]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:14 smithi012 ceph-mon[28184]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:14 smithi154 ceph-mon[32578]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:15.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.624+0000 7f35977fe700 1 -- 172.21.15.154:0/675044549 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35a00ffae0 con 0x7f358c0a3c40 2024-04-26T21:57:15.625 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 8 on host 'smithi154' 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 msgr2=0x7f358c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 0x7f358c0a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35a01033d0 tx=0x7f359c008040 comp rx=0 tx=0).stop 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 msgr2=0x7f35a0104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f359800b070 tx=0x7f3598004ef0 comp rx=0 tx=0).stop 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 shutdown_connections 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f358c0a3c40 0x7f358c0a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a0103f10 0x7f35a0104e90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:15.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a0102370 0x7f35a01062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:15.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 --2- 172.21.15.154:0/675044549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35a0103570 0x7f35a0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:15.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 >> 172.21.15.154:0/675044549 conn(0x7f35a00fd910 msgr2=0x7f35a00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:15.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 shutdown_connections 2024-04-26T21:57:15.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:57:15.627+0000 7f35a869c700 1 -- 172.21.15.154:0/675044549 wait complete. 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:15.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:16.276 DEBUG:teuthology.orchestra.run.smithi154:osd.8> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:57:16.279 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-04-26T21:57:16.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:57:16.740 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:57:16.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:57:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:57:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:16.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:16.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T21:57:16.778+0000 7f1619530700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:57:16.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 21:57:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T21:57:16.791+0000 7f1615323700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:57:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:17.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:57:17.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:17.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:57:17.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:17.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:16 smithi012 ceph-mon[28184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: from='osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:17.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:16 smithi077 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:57:17.868 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:17 smithi012 ceph-mon[28184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:57:17.868 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:17 smithi012 ceph-mon[28184]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:57:17.868 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:17 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:17.868 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:17 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:17 smithi154 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:57:18.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:17 smithi154 ceph-mon[32578]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:57:18.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:17 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:17 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:17 smithi077 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:57:18.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:17 smithi077 ceph-mon[32280]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:57:18.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:17 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:17 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] boot 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:57:18.792 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] boot 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:57:18.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:19.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:57:19.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:57:19.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:57:19.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: osd.8 [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] boot 2024-04-26T21:57:19.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:57:19.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0552768 s, 190 MB/s 2024-04-26T21:57:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:19 smithi012 ceph-mon[28184]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:19 smithi154 ceph-mon[32578]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:57:20.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:57:20.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:19 smithi077 ceph-mon[32280]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:57:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:20 smithi012 ceph-mon[28184]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:20 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi154 2024-04-26T21:57:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:20 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-26T21:57:20.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-04-26T21:57:21.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:20 smithi154 ceph-mon[32578]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:21.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:20 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi154 2024-04-26T21:57:21.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:20 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-26T21:57:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:20 smithi077 ceph-mon[32280]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:20 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi154 2024-04-26T21:57:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:20 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-26T21:57:21.456 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:57:21.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:21 smithi012 ceph-mon[28184]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:57:22.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:21 smithi154 ceph-mon[32578]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:57:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:21 smithi077 ceph-mon[32280]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:57:23.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:22 smithi012 ceph-mon[28184]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:23.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:22 smithi012 ceph-mon[28184]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:57:23.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:22 smithi154 ceph-mon[32578]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:23.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:22 smithi154 ceph-mon[32578]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:57:23.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:22 smithi077 ceph-mon[32280]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:23.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:22 smithi077 ceph-mon[32280]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 -- 172.21.15.12:0/817501414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210104680 msgr2=0x7f4210104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 --2- 172.21.15.12:0/817501414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210104680 0x7f4210104ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0099c0 tx=0x7f41fc009cd0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 -- 172.21.15.12:0/817501414 shutdown_connections 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 --2- 172.21.15.12:0/817501414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105020 0x7f421010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 --2- 172.21.15.12:0/817501414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 --2- 172.21.15.12:0/817501414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210104680 0x7f4210104ae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.556+0000 7f4214e00700 1 -- 172.21.15.12:0/817501414 >> 172.21.15.12:0/817501414 conn(0x7f42100fea20 msgr2=0x7f4210100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.557+0000 7f4214e00700 1 -- 172.21.15.12:0/817501414 shutdown_connections 2024-04-26T21:57:23.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.557+0000 7f4214e00700 1 -- 172.21.15.12:0/817501414 wait complete. 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 Processor -- start 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 -- start start 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210104680 0x7f4210199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 0x7f42101a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f421010ef20 con 0x7f4210105020 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f421010eda0 con 0x7f4210104680 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.558+0000 7f4214e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f421010f0a0 con 0x7f4210103480 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420e800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 0x7f42101a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:36770/0 (socket says 172.21.15.12:36770) 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 -- 172.21.15.12:0/3740681879 learned_addr learned my addr 172.21.15.12:0/3740681879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:57:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420d400700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210104680 0x7f4210199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 -- 172.21.15.12:0/3740681879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210104680 msgr2=0x7f4210199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210104680 0x7f4210199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 -- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 msgr2=0x7f42101a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 0x7f42101a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 -- 172.21.15.12:0/3740681879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42101a1240 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420e800700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 0x7f42101a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.559+0000 7f420de00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f420000e980 tx=0x7f420000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.560+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200004230 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.560+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42000043d0 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.560+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101a1530 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.560+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200003d40 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.560+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42101a1a20 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.561+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421006a110 con 0x7f4210103480 2024-04-26T21:57:23.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.563+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f420000c810 con 0x7f4210103480 2024-04-26T21:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.564+0000 7f4206a00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 0x7f41f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.565+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f42000cfcb0 con 0x7f4210103480 2024-04-26T21:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.565+0000 7f420d400700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 0x7f41f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.565+0000 7f420d400700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 0x7f41f40a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41fc000c00 tx=0x7f41fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.569+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f420009cfa0 con 0x7f4210103480 2024-04-26T21:57:23.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:23.813+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4210063860 con 0x7f41f40a3c40 2024-04-26T21:57:24.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:24.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:24.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:24.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:24.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:24.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:24.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:24.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:24.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:24 smithi012 ceph-mon[28184]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-26T21:57:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:24 smithi012 ceph-mon[28184]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:25.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:24 smithi154 ceph-mon[32578]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-26T21:57:25.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:24 smithi154 ceph-mon[32578]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:25.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:24 smithi077 ceph-mon[32280]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-26T21:57:25.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:24 smithi077 ceph-mon[32280]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:27.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:26 smithi154 ceph-mon[32578]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:57:27.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:26 smithi077 ceph-mon[32280]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:57:27.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:26 smithi012 ceph-mon[28184]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:57:28.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:28 smithi012 ceph-mon[28184]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:57:28.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:28 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]: dispatch 2024-04-26T21:57:28.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:28 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]': finished 2024-04-26T21:57:28.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:28 smithi012 ceph-mon[28184]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:57:28.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:28 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:29.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:28 smithi154 ceph-mon[32578]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:57:29.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:28 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]: dispatch 2024-04-26T21:57:29.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:28 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]': finished 2024-04-26T21:57:29.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:28 smithi154 ceph-mon[32578]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:57:29.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:28 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:29.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:28 smithi077 ceph-mon[32280]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:57:29.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:28 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]: dispatch 2024-04-26T21:57:29.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:28 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/579698745' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70375e11-a2c2-4512-9403-6cc31fb3409c"}]': finished 2024-04-26T21:57:29.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:28 smithi077 ceph-mon[32280]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:57:29.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:28 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:30.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:29 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3538045264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:30.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:29 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3538045264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:30.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:29 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3538045264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:57:31.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:30 smithi154 ceph-mon[32578]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:57:31.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:30 smithi077 ceph-mon[32280]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:57:31.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:30 smithi012 ceph-mon[28184]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:57:33.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:32 smithi154 ceph-mon[32578]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-26T21:57:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:32 smithi077 ceph-mon[32280]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-26T21:57:33.280 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:32 smithi012 ceph-mon[28184]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-26T21:57:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:34 smithi154 ceph-mon[32578]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:35.162 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:34 smithi012 ceph-mon[28184]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:34 smithi077 ceph-mon[32280]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:37.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:36 smithi154 ceph-mon[32578]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:36 smithi077 ceph-mon[32280]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:37.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:36 smithi012 ceph-mon[28184]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:38.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:57:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:37 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:38.214 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:57:38.214 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:37 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:38.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:57:38.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:37 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:38 smithi154 ceph-mon[32578]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:39.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:38 smithi154 ceph-mon[32578]: Deploying daemon osd.9 on smithi012 2024-04-26T21:57:39.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:38 smithi077 ceph-mon[32280]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:39.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:38 smithi077 ceph-mon[32280]: Deploying daemon osd.9 on smithi012 2024-04-26T21:57:39.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:38 smithi012 ceph-mon[28184]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:39.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:38 smithi012 ceph-mon[28184]: Deploying daemon osd.9 on smithi012 2024-04-26T21:57:40.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:39 smithi012 ceph-mon[28184]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:40.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:39 smithi077 ceph-mon[32280]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:40.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:39 smithi154 ceph-mon[32578]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:42.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:42 smithi012 ceph-mon[28184]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:42.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:42 smithi077 ceph-mon[32280]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:42.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:42 smithi154 ceph-mon[32578]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:44.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:44 smithi012 ceph-mon[28184]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:44.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:44 smithi077 ceph-mon[32280]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:44.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:44 smithi154 ceph-mon[32578]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:46 smithi077 ceph-mon[32280]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:46.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:46 smithi012 ceph-mon[28184]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:46.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:46 smithi154 ceph-mon[32578]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:47 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:47.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.530 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:47 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:47.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:47 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:48.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:48 smithi077 ceph-mon[32280]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:48 smithi012 ceph-mon[28184]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:48.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:48 smithi154 ceph-mon[32578]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:50 smithi012 ceph-mon[28184]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:50 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:50 smithi012 ceph-mon[28184]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:50 smithi012 ceph-mon[28184]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:50 smithi154 ceph-mon[32578]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:50 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:50 smithi154 ceph-mon[32578]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:50 smithi154 ceph-mon[32578]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:50 smithi077 ceph-mon[32280]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:50 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:50 smithi077 ceph-mon[32280]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:50 smithi077 ceph-mon[32280]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:57:51.275 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-04-26T21:57:51.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.274+0000 7f4206a00700 1 -- 172.21.15.12:0/3740681879 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4210063860 con 0x7f41f40a3c40 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 msgr2=0x7f41f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 0x7f41f40a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f41fc000c00 tx=0x7f41fc005fb0 comp rx=0 tx=0).stop 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 msgr2=0x7f42101995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f420000e980 tx=0x7f420000ec90 comp rx=0 tx=0).stop 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 shutdown_connections 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105020 0x7f42101a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3c40 0x7f41f40a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210103480 0x7f42101995e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 --2- 172.21.15.12:0/3740681879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210104680 0x7f4210199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:51.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 >> 172.21.15.12:0/3740681879 conn(0x7f42100fea20 msgr2=0x7f4210107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:51.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 shutdown_connections 2024-04-26T21:57:51.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:57:51.278+0000 7f4214e00700 1 -- 172.21.15.12:0/3740681879 wait complete. 2024-04-26T21:57:52.015 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:57:52.023 INFO:tasks.cephadm:Deploying osd.10 on smithi077 with /dev/vg_nvme/lv_1... 2024-04-26T21:57:52.023 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:57:52.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:51 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:57:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:57:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:51 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.271 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:57:52.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:57:52.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:57:52.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:52.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-26T21:57:52.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:51 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:52.960 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:52 smithi077 ceph-mon[32280]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:52.960 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:52 smithi077 ceph-mon[32280]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:57:52.961 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:52 smithi077 ceph-mon[32280]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:57:52.961 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:52.961 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:52 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:53.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:52 smithi012 ceph-mon[28184]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:53.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:52 smithi012 ceph-mon[28184]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:57:53.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:52 smithi012 ceph-mon[28184]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:57:53.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:53.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:52 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:52 smithi154 ceph-mon[32578]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:57:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:52 smithi154 ceph-mon[32578]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-26T21:57:53.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:52 smithi154 ceph-mon[32578]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:57:53.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:53.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:52 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:53 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:57:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:53 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:57:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:53 smithi012 ceph-mon[28184]: osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] boot 2024-04-26T21:57:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:53 smithi012 ceph-mon[28184]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:57:54.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:53 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113417 s, 925 MB/s 2024-04-26T21:57:54.090 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T21:57:54.138 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:53 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:57:54.138 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:53 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:57:54.139 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:53 smithi077 ceph-mon[32280]: osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] boot 2024-04-26T21:57:54.139 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:53 smithi077 ceph-mon[32280]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:57:54.139 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:53 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:54.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:53 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:57:54.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:53 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:57:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:53 smithi154 ceph-mon[32578]: osd.9 [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] boot 2024-04-26T21:57:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:53 smithi154 ceph-mon[32578]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:57:54.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:53 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:57:54.771 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-04-26T21:57:55.021 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:57:55.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:54 smithi154 ceph-mon[32578]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:55.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:54 smithi154 ceph-mon[32578]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:57:55.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:54 smithi077 ceph-mon[32280]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:55.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:54 smithi077 ceph-mon[32280]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:57:55.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:54 smithi012 ceph-mon[28184]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:55.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:54 smithi012 ceph-mon[28184]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:57:56.125 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:55 smithi077 ceph-mon[32280]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:56.264 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:55 smithi012 ceph-mon[28184]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:56.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:55 smithi154 ceph-mon[32578]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/2219850412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64101ed0 msgr2=0x7fcd6410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/2219850412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64101ed0 0x7fcd6410baa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00b0a0 tx=0x7fcd5c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/2219850412 shutdown_connections 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/2219850412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64101ed0 0x7fcd6410baa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/2219850412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64104ab0 0x7fcd64104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/2219850412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd64101530 0x7fcd64101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/2219850412 >> 172.21.15.77:0/2219850412 conn(0x7fcd640fd090 msgr2=0x7fcd640ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/2219850412 shutdown_connections 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.663+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/2219850412 wait complete. 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 Processor -- start 2024-04-26T21:57:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 -- start start 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 0x7fcd641041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd64101ed0 0x7fcd64102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6410ef70 con 0x7fcd64101ed0 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6410edf0 con 0x7fcd64104ab0 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd6a0ed700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6410f0f0 con 0x7fcd64101530 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59674/0 (socket says 172.21.15.77:59674) 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 -- 172.21.15.77:0/441442639 learned_addr learned my addr 172.21.15.77:0/441442639 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd637fe700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 0x7fcd641041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 -- 172.21.15.77:0/441442639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 msgr2=0x7fcd641041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 0x7fcd641041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 -- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd64101ed0 msgr2=0x7fcd64102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd64101ed0 0x7fcd64102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 -- 172.21.15.77:0/441442639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd641035f0 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd637fe700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 0x7fcd641041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.664+0000 7fcd63fff700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00ced0 tx=0x7fcd5c00cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.665+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c021070 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.665+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd5c01d900 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.665+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd641a35f0 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.665+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c005420 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.665+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a3b80 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.666+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6406a110 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.667+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd5c024070 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.667+0000 7fcd617fa700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 0x7fcd500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.667+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fcd5c00f020 con 0x7fcd64104ab0 2024-04-26T21:57:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.668+0000 7fcd637fe700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 0x7fcd500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:56.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.669+0000 7fcd637fe700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 0x7fcd500a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd6406b8b0 tx=0x7fcd5400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:56.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.672+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcd5c02b020 con 0x7fcd64104ab0 2024-04-26T21:57:56.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:57:56.880+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcd64063860 con 0x7fcd500a3c40 2024-04-26T21:57:57.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:56 smithi077 ceph-mon[32280]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:57.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:57.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:57.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:57.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:56 smithi012 ceph-mon[28184]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:57.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:57.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:57.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:57.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:56 smithi154 ceph-mon[32578]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:57:57.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:57:57.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:57:57.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi012 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:58.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:57:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi012 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:57:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi012 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:58.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:57:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/758911694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]': finished 2024-04-26T21:58:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:58:00.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/758911694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]': finished 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:58:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/758911694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]: dispatch 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a526fb3-2f0b-4161-9b40-9ba2774e847a"}]': finished 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:58:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:01 smithi077 ceph-mon[32280]: from='client.? 172.21.15.77:0/2348653044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:01 smithi012 ceph-mon[28184]: from='client.? 172.21.15.77:0/2348653044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:01 smithi154 ceph-mon[32578]: from='client.? 172.21.15.77:0/2348653044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:02 smithi077 ceph-mon[32280]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:02.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:02 smithi012 ceph-mon[28184]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:02.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:02 smithi154 ceph-mon[32578]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:04.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:04 smithi077 ceph-mon[32280]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:04.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:04 smithi012 ceph-mon[28184]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:04.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:04 smithi154 ceph-mon[32578]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:06.461 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:06 smithi077 ceph-mon[32280]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:06.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:06 smithi012 ceph-mon[28184]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:06.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:06 smithi154 ceph-mon[32578]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:07.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:07 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:58:07.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:07 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:07 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:58:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:07 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:07.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:07 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:58:07.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:07 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:08.425 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:08 smithi077 ceph-mon[32280]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:08.426 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:08 smithi077 ceph-mon[32280]: Deploying daemon osd.10 on smithi077 2024-04-26T21:58:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:08 smithi012 ceph-mon[28184]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:08 smithi012 ceph-mon[28184]: Deploying daemon osd.10 on smithi077 2024-04-26T21:58:08.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:08 smithi154 ceph-mon[32578]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:08.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:08 smithi154 ceph-mon[32578]: Deploying daemon osd.10 on smithi077 2024-04-26T21:58:10.441 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:10 smithi077 ceph-mon[32280]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:10 smithi012 ceph-mon[28184]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:10 smithi154 ceph-mon[32578]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:12.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:12 smithi077 ceph-mon[32280]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:12.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:12 smithi012 ceph-mon[28184]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:12.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:12 smithi154 ceph-mon[32578]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:13 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:13 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:13 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:14.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:13 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:13 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:13 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:14.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:13 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:13 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:14.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:13 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:14 smithi012 ceph-mon[28184]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:15.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:14 smithi154 ceph-mon[32578]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:15.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:14 smithi077 ceph-mon[32280]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:16.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.820+0000 7fcd617fa700 1 -- 172.21.15.77:0/441442639 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcd64063860 con 0x7fcd500a3c40 2024-04-26T21:58:16.821 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 10 on host 'smithi077' 2024-04-26T21:58:16.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 msgr2=0x7fcd500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 0x7fcd500a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd6406b8b0 tx=0x7fcd5400a680 comp rx=0 tx=0).stop 2024-04-26T21:58:16.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 msgr2=0x7fcd64102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00ced0 tx=0x7fcd5c00cfb0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 shutdown_connections 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd64104ab0 0x7fcd64102d40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcd500a3c40 0x7fcd500a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd64101530 0x7fcd641041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 --2- 172.21.15.77:0/441442639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd64101ed0 0x7fcd64102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 >> 172.21.15.77:0/441442639 conn(0x7fcd640fd090 msgr2=0x7fcd64107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 shutdown_connections 2024-04-26T21:58:16.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T21:58:16.822+0000 7fcd6a0ed700 1 -- 172.21.15.77:0/441442639 wait complete. 2024-04-26T21:58:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:16 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:16 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:17.193 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.194 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.194 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:58:17.194 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.194 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:16 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:17.561 DEBUG:teuthology.orchestra.run.smithi077:osd.10> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:58:17.564 INFO:tasks.cephadm:Deploying osd.11 on smithi154 with /dev/vg_nvme/lv_1... 2024-04-26T21:58:17.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:58:17.812 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:58:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:17.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:58:17.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:58:17.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:17.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:17.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:17 smithi154 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:58:18.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:58:18.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:18.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:18.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:17 smithi012 ceph-mon[28184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:18.104 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:18.105 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:58:18.105 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:58:18.105 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: from='osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:18.105 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:18.105 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:17 smithi077 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T21:58:19.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:19.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:58:19.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:58:19.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:18 smithi012 ceph-mon[28184]: from='osd.10 ' entity='osd.10' 2024-04-26T21:58:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:58:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:58:19.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-mon[32280]: from='osd.10 ' entity='osd.10' 2024-04-26T21:58:19.099 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T21:58:18.739+0000 7fd80ae69700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T21:58:19.099 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 21:58:18 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T21:58:18.752+0000 7fd802c54700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:58:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T21:58:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:58:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:19.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:18 smithi154 ceph-mon[32578]: from='osd.10 ' entity='osd.10' 2024-04-26T21:58:19.799 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:58:19.799 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:58:19.799 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-26T21:58:19.799 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-26T21:58:19.800 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115793 s, 906 MB/s 2024-04-26T21:58:19.800 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-26T21:58:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:58:20.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:58:20.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] boot 2024-04-26T21:58:20.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:58:20.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:20.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:20 smithi077 ceph-mon[32280]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:20.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:58:20.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:58:20.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] boot 2024-04-26T21:58:20.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:58:20.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:20.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:20 smithi012 ceph-mon[28184]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:20.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:58:20.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:58:20.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: osd.10 [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] boot 2024-04-26T21:58:20.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:58:20.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:58:20.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:20 smithi154 ceph-mon[32578]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:58:20.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-04-26T21:58:20.707 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:58:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:58:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:21.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:58:21.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:21.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:21.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:21.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:22 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi077 2024-04-26T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:22 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-26T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:22 smithi154 ceph-mon[32578]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:22 smithi154 ceph-mon[32578]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:22 smithi154 ceph-mon[32578]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- 172.21.15.154:0/3656204939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec1039a0 msgr2=0x7f4cec103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 --2- 172.21.15.154:0/3656204939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec1039a0 0x7f4cec103dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- 172.21.15.154:0/3656204939 shutdown_connections 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 --2- 172.21.15.154:0/3656204939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec102fc0 0x7f4cec10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 --2- 172.21.15.154:0/3656204939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec1039a0 0x7f4cec103dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 --2- 172.21.15.154:0/3656204939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec102600 0x7f4cec102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- 172.21.15.154:0/3656204939 >> 172.21.15.154:0/3656204939 conn(0x7f4cec0fd090 msgr2=0x7f4cec0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- 172.21.15.154:0/3656204939 shutdown_connections 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- 172.21.15.154:0/3656204939 wait complete. 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 Processor -- start 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.447+0000 7f4cf3662700 1 -- start start 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec102600 0x7f4cec102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec1039a0 0x7f4cec1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10de90 con 0x7f4cec102600 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10dd10 con 0x7f4cec1039a0 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf3662700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10e010 con 0x7f4cec102fc0 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53124/0 (socket says 172.21.15.154:53124) 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 -- 172.21.15.154:0/4126917083 learned_addr learned my addr 172.21.15.154:0/4126917083 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:58:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf1bff700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec1039a0 0x7f4cec1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 -- 172.21.15.154:0/4126917083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec1039a0 msgr2=0x7f4cec1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec1039a0 0x7f4cec1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 -- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec102600 msgr2=0x7f4cec102050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec102600 0x7f4cec102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 -- 172.21.15.154:0/4126917083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec1a1270 con 0x7f4cec102fc0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.448+0000 7f4cf0bfd700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00b550 tx=0x7f4cdc00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.449+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc015020 con 0x7f4cec102fc0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.449+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec1a1560 con 0x7f4cec102fc0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.449+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cdc011ce0 con 0x7f4cec102fc0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.449+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc010d30 con 0x7f4cec102fc0 2024-04-26T21:58:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.449+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec1a1b70 con 0x7f4cec102fc0 2024-04-26T21:58:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.450+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cec06a110 con 0x7f4cec102fc0 2024-04-26T21:58:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.451+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4cdc018020 con 0x7f4cec102fc0 2024-04-26T21:58:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.451+0000 7f4ce27fc700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 0x7f4cd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.452+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f4cdc0cf510 con 0x7f4cec102fc0 2024-04-26T21:58:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.452+0000 7f4cf13fe700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 0x7f4cd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.453+0000 7f4cf13fe700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 0x7f4cd80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80038c0 tx=0x7f4ce8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.456+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4cdc01f020 con 0x7f4cec102fc0 2024-04-26T21:58:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:22 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi077 2024-04-26T21:58:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:22 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-26T21:58:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:22 smithi077 ceph-mon[32280]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:58:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:22 smithi077 ceph-mon[32280]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:22 smithi077 ceph-mon[32280]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:58:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:22 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi077 2024-04-26T21:58:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:22 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 4272M 2024-04-26T21:58:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:22 smithi012 ceph-mon[28184]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:58:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:22 smithi012 ceph-mon[28184]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:22 smithi012 ceph-mon[28184]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:58:22.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:22.664+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4cec063860 con 0x7f4cd80a3c40 2024-04-26T21:58:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:23 smithi012 ceph-mon[28184]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:58:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:23 smithi012 ceph-mon[28184]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:58:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:58:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:23 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:23 smithi154 ceph-mon[32578]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:58:23.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:23 smithi154 ceph-mon[32578]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:23.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:58:23.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:58:23.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:23 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:23 smithi077 ceph-mon[32280]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:58:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:23 smithi077 ceph-mon[32280]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:58:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:58:23.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:23 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:24.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:24 smithi012 ceph-mon[28184]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:58:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:24 smithi154 ceph-mon[32578]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:58:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:24 smithi077 ceph-mon[32280]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:58:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:25 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/2002843121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:25 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:25 smithi012 ceph-mon[28184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]': finished 2024-04-26T21:58:25.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:25 smithi012 ceph-mon[28184]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:58:25.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:25.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:25 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/2002843121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:25 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:25 smithi154 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]': finished 2024-04-26T21:58:25.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:25 smithi154 ceph-mon[32578]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:58:25.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:25.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:25 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/2002843121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:25 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]: dispatch 2024-04-26T21:58:25.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:25 smithi077 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfad9b30-8b56-4b29-9722-cf3de267af70"}]': finished 2024-04-26T21:58:25.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:25 smithi077 ceph-mon[32280]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:58:25.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:26 smithi012 ceph-mon[28184]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T21:58:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:26 smithi012 ceph-mon[28184]: from='client.? 172.21.15.154:0/666224498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:26.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:26 smithi154 ceph-mon[32578]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T21:58:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:26 smithi154 ceph-mon[32578]: from='client.? 172.21.15.154:0/666224498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:26 smithi077 ceph-mon[32280]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T21:58:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:26 smithi077 ceph-mon[32280]: from='client.? 172.21.15.154:0/666224498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:58:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:28 smithi012 ceph-mon[28184]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:58:28.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:28 smithi154 ceph-mon[32578]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:58:28.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:28 smithi077 ceph-mon[32280]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:58:30.651 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:30 smithi154 ceph-mon[32578]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:58:30.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:30 smithi077 ceph-mon[32280]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:58:31.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:30 smithi012 ceph-mon[28184]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-26T21:58:32.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:32 smithi154 ceph-mon[32578]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:58:32.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:32 smithi077 ceph-mon[32280]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:58:33.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:32 smithi012 ceph-mon[28184]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-26T21:58:33.718 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:33 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:58:33.718 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:33 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:33.718 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:33 smithi154 ceph-mon[32578]: Deploying daemon osd.11 on smithi154 2024-04-26T21:58:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:33 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:58:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:33 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:33 smithi077 ceph-mon[32280]: Deploying daemon osd.11 on smithi154 2024-04-26T21:58:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:33 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:58:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:33 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:33 smithi012 ceph-mon[28184]: Deploying daemon osd.11 on smithi154 2024-04-26T21:58:34.673 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:34 smithi154 ceph-mon[32578]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:34.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:34 smithi077 ceph-mon[32280]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:35.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:34 smithi012 ceph-mon[28184]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:36.746 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:36 smithi154 ceph-mon[32578]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:36.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:36 smithi077 ceph-mon[32280]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:37.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:36 smithi012 ceph-mon[28184]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:38.858 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:38 smithi154 ceph-mon[32578]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:38.859 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:38.859 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:38.859 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:38 smithi077 ceph-mon[32280]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:38 smithi012 ceph-mon[28184]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:40.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:40 smithi154 ceph-mon[32578]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:40.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:40 smithi077 ceph-mon[32280]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:40 smithi012 ceph-mon[28184]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:42.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.456+0000 7f4ce27fc700 1 -- 172.21.15.154:0/4126917083 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4cec063860 con 0x7f4cd80a3c40 2024-04-26T21:58:42.456 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 11 on host 'smithi154' 2024-04-26T21:58:42.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 msgr2=0x7f4cd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:42.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 0x7f4cd80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80038c0 tx=0x7f4ce8005fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:42.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 msgr2=0x7f4cec19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00b550 tx=0x7f4cdc00b860 comp rx=0 tx=0).stop 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 shutdown_connections 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cec1039a0 0x7f4cec1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec102600 0x7f4cec102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4cd80a3c40 0x7f4cd80a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 --2- 172.21.15.154:0/4126917083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cec102fc0 0x7f4cec19e360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 >> 172.21.15.154:0/4126917083 conn(0x7f4cec0fd090 msgr2=0x7f4cec106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 shutdown_connections 2024-04-26T21:58:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:42.458+0000 7f4cf3662700 1 -- 172.21.15.154:0/4126917083 wait complete. 2024-04-26T21:58:42.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:42.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:42.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:42 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:42.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:42 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:58:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:42.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:42 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:43.148 DEBUG:teuthology.orchestra.run.smithi154:osd.11> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:58:43.150 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:58:43.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:58:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:58:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:43 smithi154 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:58:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:43 smithi154 ceph-mon[32578]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:43 smithi154 ceph-mon[32578]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:58:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:43 smithi154 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:43.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:43 smithi012 ceph-mon[28184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:58:43.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:43 smithi012 ceph-mon[28184]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:43.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:43 smithi012 ceph-mon[28184]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:58:43.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:43.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:43 smithi012 ceph-mon[28184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:43 smithi077 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:58:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:43 smithi077 ceph-mon[32280]: from='osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:43 smithi077 ceph-mon[32280]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:58:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:43 smithi077 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-26T21:58:44.621 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T21:58:44.340+0000 7f155c7b2700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:58:44.621 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T21:58:44.355+0000 7f155459d700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:44.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:44 smithi154 ceph-mon[32578]: from='osd.11 ' entity='osd.11' 2024-04-26T21:58:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:58:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:58:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:44 smithi077 ceph-mon[32280]: from='osd.11 ' entity='osd.11' 2024-04-26T21:58:45.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:45.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-26T21:58:45.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:58:45.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:44 smithi012 ceph-mon[28184]: from='osd.11 ' entity='osd.11' 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] boot 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:58:45.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 -- 172.21.15.12:0/3495071490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b41046b0 msgr2=0x7fb2b4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3495071490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b41046b0 0x7fb2b4104b10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00099c0 tx=0x7fb2a0009cd0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 -- 172.21.15.12:0/3495071490 shutdown_connections 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3495071490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b4105050 0x7fb2b410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3495071490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3495071490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b41046b0 0x7fb2b4104b10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 -- 172.21.15.12:0/3495071490 >> 172.21.15.12:0/3495071490 conn(0x7fb2b40fea30 msgr2=0x7fb2b4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 -- 172.21.15.12:0/3495071490 shutdown_connections 2024-04-26T21:58:45.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.862+0000 7fb2bc400700 1 -- 172.21.15.12:0/3495071490 wait complete. 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: purged_snaps scrub starts 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: purged_snaps scrub ok 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] boot 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:58:45.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.864+0000 7fb2bc400700 1 Processor -- start 2024-04-26T21:58:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.864+0000 7fb2bc400700 1 -- start start 2024-04-26T21:58:45.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 0x7fb2b4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 0x7fb2b41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410ef30 con 0x7fb2b41034b0 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410edb0 con 0x7fb2b4105050 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2bc400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410f0b0 con 0x7fb2b41046b0 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2b9a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2b9000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 0x7fb2b4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2b9000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 0x7fb2b4199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56996/0 (socket says 172.21.15.12:56996) 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2ba400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 0x7fb2b41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2b9000700 1 -- 172.21.15.12:0/3833074075 learned_addr learned my addr 172.21.15.12:0/3833074075 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.865+0000 7fb2b9a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59900/0 (socket says 172.21.15.12:59900) 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 -- 172.21.15.12:0/3833074075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 msgr2=0x7fb2b4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:45.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 0x7fb2b4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 -- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 msgr2=0x7fb2b41995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 -- 172.21.15.12:0/3833074075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b4075650 con 0x7fb2b4105050 2024-04-26T21:58:45.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2b9a00700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2ba400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 0x7fb2b41a0b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00efc0 tx=0x7fb2ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0092e0 con 0x7fb2b4105050 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac004bb0 con 0x7fb2b4105050 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b40758e0 con 0x7fb2b4105050 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.866+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac004280 con 0x7fb2b4105050 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.867+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b4075e20 con 0x7fb2b4105050 2024-04-26T21:58:45.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.868+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b406a110 con 0x7fb2b4105050 2024-04-26T21:58:45.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.870+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2ac01b030 con 0x7fb2b4105050 2024-04-26T21:58:45.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.872+0000 7fb2b2000700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 0x7fb29c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.872+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb2ac0cfab0 con 0x7fb2b4105050 2024-04-26T21:58:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.872+0000 7fb2b9a00700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 0x7fb29c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:45.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.873+0000 7fb2b9a00700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 0x7fb29c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4104510 tx=0x7fb2a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:45.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:45.876+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2ac022020 con 0x7fb2b4105050 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: purged_snaps scrub starts 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: purged_snaps scrub ok 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: osd.11 [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] boot 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:58:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:58:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.112+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb2b410a310 con 0x7fb2b4105050 2024-04-26T21:58:46.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.113+0000 7fb2b2000700 1 -- 172.21.15.12:0/3833074075 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fb2ac010040 con 0x7fb2b4105050 2024-04-26T21:58:46.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 msgr2=0x7fb29c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 0x7fb29c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4104510 tx=0x7fb2a8009260 comp rx=0 tx=0).stop 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 msgr2=0x7fb2b41a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 0x7fb2b41a0b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00efc0 tx=0x7fb2ac00c3a0 comp rx=0 tx=0).stop 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 shutdown_connections 2024-04-26T21:58:46.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2b4105050 0x7fb2b41a0b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b41034b0 0x7fb2b41995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb29c0a3cc0 0x7fb29c0a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 --2- 172.21.15.12:0/3833074075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2b41046b0 0x7fb2b4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 >> 172.21.15.12:0/3833074075 conn(0x7fb2b40fea30 msgr2=0x7fb2b41079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.117+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 shutdown_connections 2024-04-26T21:58:46.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:46.118+0000 7fb2bc400700 1 -- 172.21.15.12:0/3833074075 wait complete. 2024-04-26T21:58:46.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:46.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3833074075' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:46 smithi012 ceph-mon[28184]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3833074075' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:46 smithi154 ceph-mon[32578]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:58:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714168725,"num_in_osds":12,"osd_in_since":1714168705,"num_remapped_pgs":0} 2024-04-26T21:58:46.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:58:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:58:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3833074075' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:58:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:46.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:46.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:46.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:46 smithi077 ceph-mon[32280]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:58:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:58:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:47 smithi012 ceph-mon[28184]: Detected new or changed devices on smithi154 2024-04-26T21:58:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:47 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-26T21:58:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:47 smithi077 ceph-mon[32280]: Detected new or changed devices on smithi154 2024-04-26T21:58:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:47 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-26T21:58:48.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:47 smithi154 ceph-mon[32578]: Detected new or changed devices on smithi154 2024-04-26T21:58:48.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:47 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-26T21:58:48.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:48 smithi012 ceph-mon[28184]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:48 smithi077 ceph-mon[32280]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:49.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:48 smithi154 ceph-mon[32578]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 -- 172.21.15.12:0/1956695150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 msgr2=0x7f66a01059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 --2- 172.21.15.12:0/1956695150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01059f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f668c0099c0 tx=0x7f668c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 -- 172.21.15.12:0/1956695150 shutdown_connections 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 --2- 172.21.15.12:0/1956695150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a0105f30 0x7f66a01123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 --2- 172.21.15.12:0/1956695150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a010b620 0x7f66a010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 --2- 172.21.15.12:0/1956695150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01059f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 -- 172.21.15.12:0/1956695150 >> 172.21.15.12:0/1956695150 conn(0x7f66a0101040 msgr2=0x7f66a0103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 -- 172.21.15.12:0/1956695150 shutdown_connections 2024-04-26T21:58:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.602+0000 7f66a4c00700 1 -- 172.21.15.12:0/1956695150 wait complete. 2024-04-26T21:58:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 Processor -- start 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 -- start start 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a010b620 0x7f66a019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0115790 con 0x7f66a0105570 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0115610 con 0x7f66a0105f30 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f66a4c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0115910 con 0x7f66a010b620 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f669ea00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.604+0000 7f669f400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669ea00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:32812/0 (socket says 172.21.15.12:32812) 2024-04-26T21:58:49.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669ea00700 1 -- 172.21.15.12:0/3692763357 learned_addr learned my addr 172.21.15.12:0/3692763357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669fe00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a010b620 0x7f66a019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 -- 172.21.15.12:0/3692763357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a010b620 msgr2=0x7f66a019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a010b620 0x7f66a019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 -- 172.21.15.12:0/3692763357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 msgr2=0x7f66a01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 -- 172.21.15.12:0/3692763357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019c190 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669ea00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.605+0000 7f669f400700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01a1870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f669000e9f0 tx=0x7f669000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.606+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000c9c0 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.606+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669000cb60 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.606+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690018680 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.606+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019c480 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.606+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a8740 con 0x7f66a0105570 2024-04-26T21:58:49.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.607+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a006a110 con 0x7f66a0105570 2024-04-26T21:58:49.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.609+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6690010430 con 0x7f66a0105570 2024-04-26T21:58:49.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.611+0000 7f6697e00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 0x7f66840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.611+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f66900d0410 con 0x7f66a0105570 2024-04-26T21:58:49.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.611+0000 7f669ea00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 0x7f66840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.612+0000 7f669ea00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 0x7f66840a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66a010ac50 tx=0x7f668c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.614+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f669009d1f0 con 0x7f66a0105570 2024-04-26T21:58:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.846+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66a002e490 con 0x7f66a0105570 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.849+0000 7f6697e00700 1 -- 172.21.15.12:0/3692763357 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7f669009caf0 con 0x7f66a0105570 2024-04-26T21:58:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:58:49.850 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:58:46.347916+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 msgr2=0x7f66840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 0x7f66840a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66a010ac50 tx=0x7f668c005ab0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 msgr2=0x7f66a01a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01a1870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f669000e9f0 tx=0x7f669000ed00 comp rx=0 tx=0).stop 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 shutdown_connections 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f66840a3c40 0x7f66840a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66a010b620 0x7f66a019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66a0105570 0x7f66a01a1870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 --2- 172.21.15.12:0/3692763357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66a0105f30 0x7f66a01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 >> 172.21.15.12:0/3692763357 conn(0x7f66a0101040 msgr2=0x7f66a01027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 shutdown_connections 2024-04-26T21:58:49.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:49.852+0000 7f66a4c00700 1 -- 172.21.15.12:0/3692763357 wait complete. 2024-04-26T21:58:50.597 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:58:50.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:58:50.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:50 smithi012 ceph-mon[28184]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:50.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3692763357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:50.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:50 smithi077 ceph-mon[32280]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:50.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3692763357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:58:51.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:50 smithi154 ceph-mon[32578]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:51.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3692763357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:52.749 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:52 smithi012 ceph-mon[28184]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:52.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:52 smithi077 ceph-mon[32280]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:53.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:52 smithi154 ceph-mon[32578]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 -- 172.21.15.12:0/3925601754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410a450 msgr2=0x7f1964116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 --2- 172.21.15.12:0/3925601754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410a450 0x7f1964116910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f195c00b0a0 tx=0x7f195c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 -- 172.21.15.12:0/3925601754 shutdown_connections 2024-04-26T21:58:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 --2- 172.21.15.12:0/3925601754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410a450 0x7f1964116910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 --2- 172.21.15.12:0/3925601754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410fa40 0x7f196410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 --2- 172.21.15.12:0/3925601754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f1964109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 -- 172.21.15.12:0/3925601754 >> 172.21.15.12:0/3925601754 conn(0x7f1964076bf0 msgr2=0x7f1964079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 -- 172.21.15.12:0/3925601754 shutdown_connections 2024-04-26T21:58:53.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.256+0000 7f196a200700 1 -- 172.21.15.12:0/3925601754 wait complete. 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.257+0000 7f196a200700 1 Processor -- start 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.257+0000 7f196a200700 1 -- start start 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410a450 0x7f19641a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410fa40 0x7f19641a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1964119c80 con 0x7f196410a450 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1964119b00 con 0x7f196410fa40 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f196a200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1964119e00 con 0x7f1964109a90 2024-04-26T21:58:53.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f1963400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f1963400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47080/0 (socket says 172.21.15.12:47080) 2024-04-26T21:58:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f1963400700 1 -- 172.21.15.12:0/157397032 learned_addr learned my addr 172.21.15.12:0/157397032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f1962a00700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410a450 0x7f19641a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.258+0000 7f1963e00700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410fa40 0x7f19641a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 -- 172.21.15.12:0/157397032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410fa40 msgr2=0x7f19641a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410fa40 0x7f19641a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 -- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410a450 msgr2=0x7f19641a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410a450 0x7f19641a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 -- 172.21.15.12:0/157397032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641a5090 con 0x7f1964109a90 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f1963400700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9a0 tx=0x7f195400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195400ca10 con 0x7f1964109a90 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641a5380 con 0x7f1964109a90 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195400cbb0 con 0x7f1964109a90 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954018680 con 0x7f1964109a90 2024-04-26T21:58:53.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.259+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19641acc00 con 0x7f1964109a90 2024-04-26T21:58:53.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.260+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196406a110 con 0x7f1964109a90 2024-04-26T21:58:53.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.262+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1954010430 con 0x7f1964109a90 2024-04-26T21:58:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.264+0000 7f195be00700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 0x7f19480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.264+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f19540d00e0 con 0x7f1964109a90 2024-04-26T21:58:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.264+0000 7f1962a00700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 0x7f19480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.265+0000 7f1962a00700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 0x7f19480a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19640792e0 tx=0x7f1950009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.269+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f195409cec0 con 0x7f1964109a90 2024-04-26T21:58:53.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.499+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f196402e490 con 0x7f1964109a90 2024-04-26T21:58:53.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.500+0000 7f195be00700 1 -- 172.21.15.12:0/157397032 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f195409c7c0 con 0x7f1964109a90 2024-04-26T21:58:53.501 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T21:58:53.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 msgr2=0x7f19480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 0x7f19480a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19640792e0 tx=0x7f1950009260 comp rx=0 tx=0).stop 2024-04-26T21:58:53.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 msgr2=0x7f19641a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9a0 tx=0x7f195400ecb0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 shutdown_connections 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f196410fa40 0x7f19641a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f19480a3c40 0x7f19480a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964109a90 0x7f19641a00d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 --2- 172.21.15.12:0/157397032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196410a450 0x7f19641a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.504+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 >> 172.21.15.12:0/157397032 conn(0x7f1964076bf0 msgr2=0x7f1964077460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.505+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 shutdown_connections 2024-04-26T21:58:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:53.505+0000 7f196a200700 1 -- 172.21.15.12:0/157397032 wait complete. 2024-04-26T21:58:53.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:53 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/157397032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:53.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:53 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/157397032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:54.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:53 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/157397032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:54.372 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.b on smithi077 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.e on smithi077 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.h on smithi077 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.k on smithi077 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.c on smithi154 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.f on smithi154 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.i on smithi154 2024-04-26T21:58:54.373 INFO:tasks.cephadm:Adding mds.l on smithi154 2024-04-26T21:58:54.373 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l' 2024-04-26T21:58:54.621 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:58:54.926 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:54 smithi154 ceph-mon[32578]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:54.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:54 smithi077 ceph-mon[32280]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:55.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:54 smithi012 ceph-mon[28184]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.212+0000 7f8fdd959700 1 -- 172.21.15.154:0/3505351266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 msgr2=0x7f8fd8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.212+0000 7f8fdd959700 1 --2- 172.21.15.154:0/3505351266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8108c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009990 tx=0x7f8fcc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 -- 172.21.15.154:0/3505351266 shutdown_connections 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 --2- 172.21.15.154:0/3505351266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd8109170 0x7f8fd810b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 --2- 172.21.15.154:0/3505351266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fd8103f10 0x7f8fd8106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 --2- 172.21.15.154:0/3505351266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8108c30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 -- 172.21.15.154:0/3505351266 >> 172.21.15.154:0/3505351266 conn(0x7f8fd80fd940 msgr2=0x7f8fd80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 -- 172.21.15.154:0/3505351266 shutdown_connections 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 -- 172.21.15.154:0/3505351266 wait complete. 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 Processor -- start 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.213+0000 7f8fdd959700 1 -- start start 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd8103f10 0x7f8fd81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fd8109170 0x7f8fd81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810f000 con 0x7f8fd8103f10 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810ee80 con 0x7f8fd8109170 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810f180 con 0x7f8fd8106840 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52268/0 (socket says 172.21.15.154:52268) 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 -- 172.21.15.154:0/470439998 learned_addr learned my addr 172.21.15.154:0/470439998 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 -- 172.21.15.154:0/470439998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fd8109170 msgr2=0x7f8fd81a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fd8109170 0x7f8fd81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 -- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd8103f10 msgr2=0x7f8fd81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd8103f10 0x7f8fd81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 -- 172.21.15.154:0/470439998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81a1340 con 0x7f8fd8106840 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fd67fc700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0038a0 tx=0x7f8fcc0039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc027070 con 0x7f8fd8106840 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a15d0 con 0x7f8fd8106840 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fcc022470 con 0x7f8fd8106840 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc00f6c0 con 0x7f8fd8106840 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.214+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a1b00 con 0x7f8fd8106840 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.215+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd806a110 con 0x7f8fd8106840 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.216+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fcc02a070 con 0x7f8fd8106840 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.217+0000 7f8fdc957700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 0x7f8fc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.217+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fcc0d1310 con 0x7f8fd8106840 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.217+0000 7f8fd6ffd700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 0x7f8fc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.218+0000 7f8fd6ffd700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 0x7f8fc00a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80097b0 tx=0x7f8fc8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.220+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8fcc013070 con 0x7f8fd8106840 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.428+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}) -- 0x7f8fd8063b20 con 0x7f8fc00a3c40 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.445+0000 7f8fdc957700 1 -- 172.21.15.154:0/470439998 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8fd8063b20 con 0x7f8fc00a3c40 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mds.all update... 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.447+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 msgr2=0x7f8fc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.447+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 0x7f8fc00a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80097b0 tx=0x7f8fc8006d20 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.447+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 msgr2=0x7f8fd8199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.447+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0038a0 tx=0x7f8fcc0039f0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 shutdown_connections 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fd8109170 0x7f8fd81a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd8103f10 0x7f8fd81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8fc00a3c40 0x7f8fc00a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 --2- 172.21.15.154:0/470439998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8106840 0x7f8fd8199c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 >> 172.21.15.154:0/470439998 conn(0x7f8fd80fd940 msgr2=0x7f8fd81047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 shutdown_connections 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-26T21:58:56.448+0000 7f8fdd959700 1 -- 172.21.15.154:0/470439998 wait complete. 2024-04-26T21:58:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:56 smithi077 ceph-mon[32280]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:56 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:56 smithi154 ceph-mon[32578]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:56 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:56 smithi012 ceph-mon[28184]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:57.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:56 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:57.094 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:58:57.102 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:58:57.147 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:58:57.193 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:58:57.239 DEBUG:teuthology.orchestra.run.smithi077:mds.b> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:58:57.242 DEBUG:teuthology.orchestra.run.smithi077:mds.e> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:58:57.285 DEBUG:teuthology.orchestra.run.smithi077:mds.h> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:58:57.328 DEBUG:teuthology.orchestra.run.smithi077:mds.k> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:58:57.370 DEBUG:teuthology.orchestra.run.smithi154:mds.c> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:58:57.373 DEBUG:teuthology.orchestra.run.smithi154:mds.f> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:58:57.416 DEBUG:teuthology.orchestra.run.smithi154:mds.i> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:58:57.459 DEBUG:teuthology.orchestra.run.smithi154:mds.l> sudo journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:58:57.501 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:58:57.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:58:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 -- 172.21.15.12:0/3256036854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1039a0 msgr2=0x7f673c103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 --2- 172.21.15.12:0/3256036854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1039a0 0x7f673c103e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6728009900 tx=0x7f672802a410 comp rx=0 tx=0).stop 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 -- 172.21.15.12:0/3256036854 shutdown_connections 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 --2- 172.21.15.12:0/3256036854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 0x7f673c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 --2- 172.21.15.12:0/3256036854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1039a0 0x7f673c103e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 --2- 172.21.15.12:0/3256036854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1027b0 0x7f673c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 -- 172.21.15.12:0/3256036854 >> 172.21.15.12:0/3256036854 conn(0x7f673c0fdd40 msgr2=0x7f673c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.759+0000 7f6742600640 1 -- 172.21.15.12:0/3256036854 shutdown_connections 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.760+0000 7f6742600640 1 -- 172.21.15.12:0/3256036854 wait complete. 2024-04-26T21:58:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.761+0000 7f6742600640 1 Processor -- start 2024-04-26T21:58:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.761+0000 7f6742600640 1 -- start start 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 0x7f673c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 0x7f673c1a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e310 con 0x7f673c1027b0 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e190 con 0x7f673c104360 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6742600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e490 con 0x7f673c1039a0 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6741200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6741200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49052/0 (socket says 172.21.15.12:49052) 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f673be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 0x7f673c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:57.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6741200640 1 -- 172.21.15.12:0/1217833367 learned_addr learned my addr 172.21.15.12:0/1217833367 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.762+0000 7f6741c00640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 0x7f673c1a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:57.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 -- 172.21.15.12:0/1217833367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 msgr2=0x7f673c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 0x7f673c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 -- 172.21.15.12:0/1217833367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 msgr2=0x7f673c1a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 0x7f673c1a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 -- 172.21.15.12:0/1217833367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a2780 con 0x7f673c1027b0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f673be00640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 0x7f673c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6741200640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f673000e9c0 tx=0x7f673000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673000cd80 con 0x7f673c1027b0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67300035f0 con 0x7f673c1027b0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730010690 con 0x7f673c1027b0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a2a70 con 0x7f673c1027b0 2024-04-26T21:58:57.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.763+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a2fe0 con 0x7f673c1027b0 2024-04-26T21:58:57.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.765+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c1a3270 con 0x7f673c1027b0 2024-04-26T21:58:57.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.768+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67300037a0 con 0x7f673c1027b0 2024-04-26T21:58:57.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.769+0000 7f6739600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 0x7f67240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:57.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.769+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f67300d03f0 con 0x7f673c1027b0 2024-04-26T21:58:57.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.769+0000 7f673be00640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 0x7f67240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:57.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.770+0000 7f673be00640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 0x7f67240a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6728031500 tx=0x7f6728005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:57.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.772+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f673009d1d0 con 0x7f673c1027b0 2024-04-26T21:58:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:57 smithi077 ceph-mon[32280]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:57 smithi077 ceph-mon[32280]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-26T21:58:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:57.999+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f673c1a3440 con 0x7f673c1027b0 2024-04-26T21:58:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.003+0000 7f6739600640 1 -- 172.21.15.12:0/1217833367 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7f673009cad0 con 0x7f673c1027b0 2024-04-26T21:58:58.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:58:58.004 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:58:46.347916+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 msgr2=0x7f67240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 0x7f67240a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6728031500 tx=0x7f6728005a20 comp rx=0 tx=0).stop 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 msgr2=0x7f673c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f673000e9c0 tx=0x7f673000ee80 comp rx=0 tx=0).stop 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 shutdown_connections 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f67240a3c30 0x7f67240a60f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f673c104360 0x7f673c1a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c1039a0 0x7f673c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 --2- 172.21.15.12:0/1217833367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f673c1027b0 0x7f673c19ab60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 >> 172.21.15.12:0/1217833367 conn(0x7f673c0fdd40 msgr2=0x7f673c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:58.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 shutdown_connections 2024-04-26T21:58:58.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.006+0000 7f6742600640 1 -- 172.21.15.12:0/1217833367 wait complete. 2024-04-26T21:58:58.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:57 smithi012 ceph-mon[28184]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:58.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:57 smithi012 ceph-mon[28184]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-26T21:58:58.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:58:58.028 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:58:58.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:57 smithi154 ceph-mon[32578]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:58:58.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:57 smithi154 ceph-mon[32578]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 -- 172.21.15.12:0/869942622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10a180 msgr2=0x7fd72c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 --2- 172.21.15.12:0/869942622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10a180 0x7fd72c10a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd714009be0 tx=0x7fd71402a6d0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 -- 172.21.15.12:0/869942622 shutdown_connections 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 --2- 172.21.15.12:0/869942622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10ab40 0x7fd72c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 --2- 172.21.15.12:0/869942622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10a180 0x7fd72c10a600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 --2- 172.21.15.12:0/869942622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.230+0000 7fd732400640 1 -- 172.21.15.12:0/869942622 >> 172.21.15.12:0/869942622 conn(0x7fd72c077f30 msgr2=0x7fd72c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.231+0000 7fd732400640 1 -- 172.21.15.12:0/869942622 shutdown_connections 2024-04-26T21:58:58.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.231+0000 7fd732400640 1 -- 172.21.15.12:0/869942622 wait complete. 2024-04-26T21:58:58.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 Processor -- start 2024-04-26T21:58:58.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 -- start start 2024-04-26T21:58:58.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10a180 0x7fd72c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10ab40 0x7fd72c1a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114ad0 con 0x7fd72c10a180 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114950 con 0x7fd72c10ab40 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.232+0000 7fd732400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114c50 con 0x7fd72c108f90 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47122/0 (socket says 172.21.15.12:47122) 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 -- 172.21.15.12:0/4028343160 learned_addr learned my addr 172.21.15.12:0/4028343160 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72aa00640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10a180 0x7fd72c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72be00640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10ab40 0x7fd72c1a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 -- 172.21.15.12:0/4028343160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10ab40 msgr2=0x7fd72c1a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10ab40 0x7fd72c1a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 -- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10a180 msgr2=0x7fd72c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10a180 0x7fd72c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 -- 172.21.15.12:0/4028343160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1a6db0 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.233+0000 7fd72b400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00e9f0 tx=0x7fd71c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.234+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00ccc0 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.234+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd71c0035f0 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.234+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00ce60 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.234+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1a70a0 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.234+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1a7570 con 0x7fd72c108f90 2024-04-26T21:58:58.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.235+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c110350 con 0x7fd72c108f90 2024-04-26T21:58:58.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.237+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd71c004000 con 0x7fd72c108f90 2024-04-26T21:58:58.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.239+0000 7fd723e00640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 0x7fd70c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.239+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd71c0d01c0 con 0x7fd72c108f90 2024-04-26T21:58:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.239+0000 7fd72aa00640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 0x7fd70c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.240+0000 7fd72aa00640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 0x7fd70c0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71402acc0 tx=0x7fd714005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:58.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.242+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd71c09cfa0 con 0x7fd72c108f90 2024-04-26T21:58:58.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.445+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd72c02dd80 con 0x7fd72c108f90 2024-04-26T21:58:58.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.446+0000 7fd723e00640 1 -- 172.21.15.12:0/4028343160 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd71c09c8a0 con 0x7fd72c108f90 2024-04-26T21:58:58.446 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T21:58:58.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.448+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 msgr2=0x7fd70c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.448+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 0x7fd70c0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71402acc0 tx=0x7fd714005fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.448+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 msgr2=0x7fd72c19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.448+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00e9f0 tx=0x7fd71c00eeb0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 shutdown_connections 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd70c0a3c30 0x7fd70c0a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c10ab40 0x7fd72c1a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd72c10a180 0x7fd72c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 --2- 172.21.15.12:0/4028343160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd72c108f90 0x7fd72c19f190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 >> 172.21.15.12:0/4028343160 conn(0x7fd72c077f30 msgr2=0x7fd72c10d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 shutdown_connections 2024-04-26T21:58:58.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.449+0000 7fd732400640 1 -- 172.21.15.12:0/4028343160 wait complete. 2024-04-26T21:58:58.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:58:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 -- 172.21.15.12:0/1359355972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 msgr2=0x7fd4e8074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/1359355972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e8074d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009900 tx=0x7fd4d802a480 comp rx=0 tx=0).stop 2024-04-26T21:58:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 -- 172.21.15.12:0/1359355972 shutdown_connections 2024-04-26T21:58:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/1359355972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 0x7fd4e810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/1359355972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e8075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/1359355972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e8074d60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 -- 172.21.15.12:0/1359355972 >> 172.21.15.12:0/1359355972 conn(0x7fd4e80fdd10 msgr2=0x7fd4e8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 -- 172.21.15.12:0/1359355972 shutdown_connections 2024-04-26T21:58:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.694+0000 7fd4e7e00640 1 -- 172.21.15.12:0/1359355972 wait complete. 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.695+0000 7fd4e7e00640 1 Processor -- start 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 -- start start 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 0x7fd4e81a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81104f0 con 0x7fd4e810a630 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8110370 con 0x7fd4e8076910 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8110670 con 0x7fd4e8075330 2024-04-26T21:58:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47132/0 (socket says 172.21.15.12:47132) 2024-04-26T21:58:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e6a00640 1 -- 172.21.15.12:0/3724692 learned_addr learned my addr 172.21.15.12:0/3724692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:58:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e7400640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 0x7fd4e81a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.696+0000 7fd4e6000640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 -- 172.21.15.12:0/3724692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 msgr2=0x7fd4e819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 -- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 msgr2=0x7fd4e81a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 0x7fd4e81a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 -- 172.21.15.12:0/3724692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a27f0 con 0x7fd4e8076910 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6a00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e6000640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e819b160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9f0 tx=0x7fd4d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400cd80 con 0x7fd4e8076910 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d40035f0 con 0x7fd4e8076910 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a2ae0 con 0x7fd4e8076910 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.697+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4010690 con 0x7fd4e8076910 2024-04-26T21:58:58.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.698+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81a3080 con 0x7fd4e8076910 2024-04-26T21:58:58.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.699+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e81093f0 con 0x7fd4e8076910 2024-04-26T21:58:58.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.701+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4d40037a0 con 0x7fd4e8076910 2024-04-26T21:58:58.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.702+0000 7fd4df400640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 0x7fd4cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:58.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.703+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd4d40d0230 con 0x7fd4e8076910 2024-04-26T21:58:58.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.703+0000 7fd4e6a00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 0x7fd4cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:58.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.703+0000 7fd4e6a00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 0x7fd4cc0a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80096c0 tx=0x7fd4d8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:58.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.707+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd4d409d010 con 0x7fd4e8076910 2024-04-26T21:58:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:58.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: Deploying daemon mds.l on smithi154 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1217833367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:58.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:58 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4028343160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:58.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.960+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e8063530 con 0x7fd4e8076910 2024-04-26T21:58:58.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.961+0000 7fd4df400640 1 -- 172.21.15.12:0/3724692 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fd4d409c910 con 0x7fd4e8076910 2024-04-26T21:58:58.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:58:58.961 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:58:58.962 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-04-26T21:58:58.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 msgr2=0x7fd4cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 0x7fd4cc0a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80096c0 tx=0x7fd4d8005a20 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 msgr2=0x7fd4e819b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e819b160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9f0 tx=0x7fd4d400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 shutdown_connections 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4cc0a3c30 0x7fd4cc0a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4e810a630 0x7fd4e81a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e8076910 0x7fd4e819b160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 --2- 172.21.15.12:0/3724692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4e8075330 0x7fd4e819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:58.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 >> 172.21.15.12:0/3724692 conn(0x7fd4e80fdd10 msgr2=0x7fd4e81065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 shutdown_connections 2024-04-26T21:58:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:58:58.964+0000 7fd4e7e00640 1 -- 172.21.15.12:0/3724692 wait complete. 2024-04-26T21:58:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:58.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: Deploying daemon mds.l on smithi154 2024-04-26T21:58:58.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1217833367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:58.984 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:58 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4028343160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: Deploying daemon mds.l on smithi154 2024-04-26T21:58:59.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1217833367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:59.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:58 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4028343160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:59.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:58:59 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3724692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:00.063 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:58:59 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3724692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:00.063 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:58:59 smithi154 systemd[1]: Starting Ceph mds.l for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:00.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:58:59 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3724692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:00.323 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 podman[62501]: 2024-04-26 21:58:59.966508075 +0000 UTC m=+0.018186807 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:00.323 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 podman[62501]: 2024-04-26 21:59:00.202988285 +0000 UTC m=+0.254667004 container create 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 podman[62501]: 2024-04-26 21:59:00.430894829 +0000 UTC m=+0.482573547 container init 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 podman[62501]: 2024-04-26 21:59:00.433850821 +0000 UTC m=+0.485529541 container start 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mds[62520]: -- 172.21.15.154:0/1179119443 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581d9be3a40 con 0x5581d9bb6800 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l[62516]: starting mds.l at 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 bash[62501]: 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 2024-04-26T21:59:00.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 21:59:00 smithi154 systemd[1]: Started Ceph mds.l for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.154:6832/1839140447,v1:172.21.15.154:6833/1839140447] up:boot 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: fsmap 1 up:standby 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:00.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:00.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:00.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:00.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:00 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:00.986 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.154:6832/1839140447,v1:172.21.15.154:6833/1839140447] up:boot 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: fsmap 1 up:standby 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.068 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:01.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:01.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:00 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.154:6832/1839140447,v1:172.21.15.154:6833/1839140447] up:boot 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: fsmap 1 up:standby 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:01.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:00 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.211+0000 7f23abe00640 1 -- 172.21.15.12:0/2000066335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac10a940 msgr2=0x7f23ac1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.211+0000 7f23abe00640 1 --2- 172.21.15.12:0/2000066335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac10a940 0x7f23ac1111d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ca0 tx=0x7f23a402a8b0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 -- 172.21.15.12:0/2000066335 shutdown_connections 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 --2- 172.21.15.12:0/2000066335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac10a940 0x7f23ac1111d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 --2- 172.21.15.12:0/2000066335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac109f80 0x7f23ac10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 --2- 172.21.15.12:0/2000066335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 0x7f23ac1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 -- 172.21.15.12:0/2000066335 >> 172.21.15.12:0/2000066335 conn(0x7f23ac077d30 msgr2=0x7f23ac07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 -- 172.21.15.12:0/2000066335 shutdown_connections 2024-04-26T21:59:01.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.212+0000 7f23abe00640 1 -- 172.21.15.12:0/2000066335 wait complete. 2024-04-26T21:59:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 Processor -- start 2024-04-26T21:59:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 -- start start 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 0x7f23ac19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac10a940 0x7f23ac1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac114910 con 0x7f23ac109f80 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac114790 con 0x7f23ac10a940 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.213+0000 7f23abe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac114a90 con 0x7f23ac108d90 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52232/0 (socket says 172.21.15.12:52232) 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 -- 172.21.15.12:0/4204566008 learned_addr learned my addr 172.21.15.12:0/4204566008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23aaa00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 0x7f23ac19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:01.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23ab400640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac10a940 0x7f23ac1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:01.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 -- 172.21.15.12:0/4204566008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 msgr2=0x7f23ac19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:01.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 0x7f23ac19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 -- 172.21.15.12:0/4204566008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac10a940 msgr2=0x7f23ac1a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac10a940 0x7f23ac1a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.214+0000 7f23a3e00640 1 -- 172.21.15.12:0/4204566008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac1a6b40 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23a3e00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f239400e9f0 tx=0x7f239400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400cd80 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23940108e0 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac1a6e30 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394018980 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.215+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1a7340 con 0x7f23ac109f80 2024-04-26T21:59:01.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.216+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac1100b0 con 0x7f23ac109f80 2024-04-26T21:59:01.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.218+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2394010430 con 0x7f23ac109f80 2024-04-26T21:59:01.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.219+0000 7f23a3400640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 0x7f23900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:01.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.220+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23940d0260 con 0x7f23ac109f80 2024-04-26T21:59:01.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.220+0000 7f23aaa00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 0x7f23900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:01.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.221+0000 7f23aaa00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 0x7f23900a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23ac109de0 tx=0x7f239c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:01.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.224+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f239409c9f0 con 0x7f23ac109f80 2024-04-26T21:59:01.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.475+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23ac0635d0 con 0x7f23ac109f80 2024-04-26T21:59:01.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.476+0000 7f23a3400640 1 -- 172.21.15.12:0/4204566008 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f23940a0cf0 con 0x7f23ac109f80 2024-04-26T21:59:01.476 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:01.476 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:01.476 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.478+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 msgr2=0x7f23900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.478+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 0x7f23900a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23ac109de0 tx=0x7f239c005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.478+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 msgr2=0x7f23ac19f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.478+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f239400e9f0 tx=0x7f239400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 shutdown_connections 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f23900a3c30 0x7f23900a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23ac10a940 0x7f23ac1a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23ac109f80 0x7f23ac19f460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 --2- 172.21.15.12:0/4204566008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23ac108d90 0x7f23ac19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 >> 172.21.15.12:0/4204566008 conn(0x7f23ac077d30 msgr2=0x7f23ac10d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 shutdown_connections 2024-04-26T21:59:01.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:01.479+0000 7f23abe00640 1 -- 172.21.15.12:0/4204566008 wait complete. 2024-04-26T21:59:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:01 smithi012 ceph-mon[28184]: Deploying daemon mds.k on smithi077 2024-04-26T21:59:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:01 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4204566008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:01 smithi077 ceph-mon[32280]: Deploying daemon mds.k on smithi077 2024-04-26T21:59:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:01 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4204566008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:02.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:01 smithi154 ceph-mon[32578]: Deploying daemon mds.k on smithi077 2024-04-26T21:59:02.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:01 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4204566008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:02.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:02 smithi077 ceph-mon[32280]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:02.983 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:02 smithi077 systemd[1]: Starting Ceph mds.k for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:03.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:02 smithi012 ceph-mon[28184]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:03.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:02 smithi154 ceph-mon[32578]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:03.351 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 podman[61322]: 2024-04-26 21:59:02.983060283 +0000 UTC m=+0.017840807 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:03.351 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 podman[61322]: 2024-04-26 21:59:03.105714486 +0000 UTC m=+0.140494982 container create 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T21:59:03.351 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 podman[61322]: 2024-04-26 21:59:03.325156022 +0000 UTC m=+0.359936529 container init 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:59:03.351 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 podman[61322]: 2024-04-26 21:59:03.328637966 +0000 UTC m=+0.363418470 container start 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:59:03.499 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 -- 172.21.15.12:0/2591523289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10106720 msgr2=0x7f8e1010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:03.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 --2- 172.21.15.12:0/2591523289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10106720 0x7f8e1010cfb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009900 tx=0x7f8e0802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 -- 172.21.15.12:0/2591523289 shutdown_connections 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 --2- 172.21.15.12:0/2591523289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10106720 0x7f8e1010cfb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 --2- 172.21.15.12:0/2591523289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10105d60 0x7f8e101061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 --2- 172.21.15.12:0/2591523289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10104b70 0x7f8e10104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 -- 172.21.15.12:0/2591523289 >> 172.21.15.12:0/2591523289 conn(0x7f8e10100120 msgr2=0x7f8e10102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 -- 172.21.15.12:0/2591523289 shutdown_connections 2024-04-26T21:59:03.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.705+0000 7f8e16400640 1 -- 172.21.15.12:0/2591523289 wait complete. 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.706+0000 7f8e16400640 1 Processor -- start 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.706+0000 7f8e16400640 1 -- start start 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10104b70 0x7f8e1019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10105d60 0x7f8e1019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:03.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101106d0 con 0x7f8e10104b70 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10110550 con 0x7f8e10106720 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e16400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10110850 con 0x7f8e10105d60 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10104b70 0x7f8e1019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48710/0 (socket says 172.21.15.12:48710) 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 -- 172.21.15.12:0/1060773703 learned_addr learned my addr 172.21.15.12:0/1060773703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e06a00640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10105d60 0x7f8e1019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 -- 172.21.15.12:0/1060773703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10105d60 msgr2=0x7f8e1019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10105d60 0x7f8e1019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 -- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10104b70 msgr2=0x7f8e1019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10104b70 0x7f8e1019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.707+0000 7f8e0fe00640 1 -- 172.21.15.12:0/1060773703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e101a2970 con 0x7f8e10106720 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e0fe00640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005ab0 tx=0x7f8e08003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0803f070 con 0x7f8e10106720 2024-04-26T21:59:03.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e08031c50 con 0x7f8e10106720 2024-04-26T21:59:03.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08035030 con 0x7f8e10106720 2024-04-26T21:59:03.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e101a2c00 con 0x7f8e10106720 2024-04-26T21:59:03.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.708+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a3140 con 0x7f8e10106720 2024-04-26T21:59:03.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.710+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1010be90 con 0x7f8e10106720 2024-04-26T21:59:03.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.711+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e08042070 con 0x7f8e10106720 2024-04-26T21:59:03.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.713+0000 7f8e0cc00640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 0x7f8df40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.713+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e080f0610 con 0x7f8e10106720 2024-04-26T21:59:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.713+0000 7f8e0f400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 0x7f8df40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.714+0000 7f8e0f400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 0x7f8df40a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e10105bc0 tx=0x7f8e00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:03.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.717+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e080bcda0 con 0x7f8e10106720 2024-04-26T21:59:03.732 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 ceph-mds[61341]: -- 172.21.15.77:0/856091598 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1e89a5a40 con 0x55b1e8978800 2024-04-26T21:59:03.732 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k[61337]: starting mds.k at 2024-04-26T21:59:03.732 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 bash[61322]: 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 2024-04-26T21:59:03.732 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 21:59:03 smithi077 systemd[1]: Started Ceph mds.k for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.965+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e100635d0 con 0x7f8e10106720 2024-04-26T21:59:03.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.966+0000 7f8e0cc00640 1 -- 172.21.15.12:0/1060773703 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f8e080c10a0 con 0x7f8e10106720 2024-04-26T21:59:03.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:03.966 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 msgr2=0x7f8df40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 0x7f8df40a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e10105bc0 tx=0x7f8e00009500 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 msgr2=0x7f8e101a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005ab0 tx=0x7f8e08003b90 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 shutdown_connections 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8df40a3c30 0x7f8df40a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e10106720 0x7f8e101a2300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e10105d60 0x7f8e1019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.968+0000 7f8e16400640 1 --2- 172.21.15.12:0/1060773703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e10104b70 0x7f8e1019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.969+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 >> 172.21.15.12:0/1060773703 conn(0x7f8e10100120 msgr2=0x7f8e10109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.969+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 shutdown_connections 2024-04-26T21:59:03.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:03.969+0000 7f8e16400640 1 -- 172.21.15.12:0/1060773703 wait complete. 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: Deploying daemon mds.j on smithi012 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6832/432973357,v1:172.21.15.77:6833/432973357] up:boot 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: fsmap 2 up:standby 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:04.725 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:04 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1060773703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:04.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: Deploying daemon mds.j on smithi012 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6832/432973357,v1:172.21.15.77:6833/432973357] up:boot 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: fsmap 2 up:standby 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:04.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:04 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1060773703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: Deploying daemon mds.j on smithi012 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6832/432973357,v1:172.21.15.77:6833/432973357] up:boot 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: fsmap 2 up:standby 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:04.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:04 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1060773703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:05.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:06.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.221+0000 7f43f0400640 1 -- 172.21.15.12:0/136408038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e810a290 msgr2=0x7f43e8110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.221+0000 7f43f0400640 1 --2- 172.21.15.12:0/136408038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e810a290 0x7f43e8110c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009900 tx=0x7f43d402a400 comp rx=0 tx=0).stop 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.222+0000 7f43f0400640 1 -- 172.21.15.12:0/136408038 shutdown_connections 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.222+0000 7f43f0400640 1 --2- 172.21.15.12:0/136408038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e81111b0 0x7f43e81135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.222+0000 7f43f0400640 1 --2- 172.21.15.12:0/136408038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e810a290 0x7f43e8110c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.222+0000 7f43f0400640 1 --2- 172.21.15.12:0/136408038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e8109930 0x7f43e8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.222+0000 7f43f0400640 1 -- 172.21.15.12:0/136408038 >> 172.21.15.12:0/136408038 conn(0x7f43e806f7c0 msgr2=0x7f43e8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:06.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.223+0000 7f43f0400640 1 -- 172.21.15.12:0/136408038 shutdown_connections 2024-04-26T21:59:06.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.223+0000 7f43f0400640 1 -- 172.21.15.12:0/136408038 wait complete. 2024-04-26T21:59:06.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.224+0000 7f43f0400640 1 Processor -- start 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 -- start start 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 0x7f43e81a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 0x7f43e81aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8116ce0 con 0x7f43e810a290 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8116b60 con 0x7f43e8109930 2024-04-26T21:59:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43f0400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8116e60 con 0x7f43e81111b0 2024-04-26T21:59:06.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43eda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:06.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.225+0000 7f43eda00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48732/0 (socket says 172.21.15.12:48732) 2024-04-26T21:59:06.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.226+0000 7f43eda00640 1 -- 172.21.15.12:0/4180998438 learned_addr learned my addr 172.21.15.12:0/4180998438 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.226+0000 7f43ed000640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 0x7f43e81a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.226+0000 7f43ee400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 0x7f43e81aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 -- 172.21.15.12:0/4180998438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 msgr2=0x7f43e81aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 0x7f43e81aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 -- 172.21.15.12:0/4180998438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 msgr2=0x7f43e81a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 -- 172.21.15.12:0/4180998438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e81ab140 con 0x7f43e810a290 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43eda00640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ee400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 0x7f43e81aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.227+0000 7f43ed000640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 0x7f43e81a3a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43d402a9f0 tx=0x7f43d4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.228+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d403f070 con 0x7f43e810a290 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.228+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d4003e70 con 0x7f43e810a290 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.228+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4035030 con 0x7f43e810a290 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.228+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81ab3d0 con 0x7f43e810a290 2024-04-26T21:59:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.228+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81ab840 con 0x7f43e810a290 2024-04-26T21:59:06.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.230+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e81abaa0 con 0x7f43e810a290 2024-04-26T21:59:06.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.233+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43d4042070 con 0x7f43e810a290 2024-04-26T21:59:06.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.235+0000 7f43e6000640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 0x7f43d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:06.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.235+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43d40f07b0 con 0x7f43e810a290 2024-04-26T21:59:06.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.236+0000 7f43eda00640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 0x7f43d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:06.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.236+0000 7f43eda00640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 0x7f43d00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00f4d0 tx=0x7f43dc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:06.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.239+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f43d40bcf40 con 0x7f43e810a290 2024-04-26T21:59:06.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.489+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e81abc70 con 0x7f43e810a290 2024-04-26T21:59:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.490+0000 7f43e6000640 1 -- 172.21.15.12:0/4180998438 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f43d40c1240 con 0x7f43e810a290 2024-04-26T21:59:06.490 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:06.490 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:06.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-04-26T21:59:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 msgr2=0x7f43d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 0x7f43d00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00f4d0 tx=0x7f43dc00a3b0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 msgr2=0x7f43e81a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 0x7f43e81a3a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43d402a9f0 tx=0x7f43d4003c80 comp rx=0 tx=0).stop 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 shutdown_connections 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f43d00a3bb0 0x7f43d00a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e81111b0 0x7f43e81aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43e810a290 0x7f43e81a3a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 --2- 172.21.15.12:0/4180998438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43e8109930 0x7f43e81a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 >> 172.21.15.12:0/4180998438 conn(0x7f43e806f7c0 msgr2=0x7f43e810aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.492+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 shutdown_connections 2024-04-26T21:59:06.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:06.493+0000 7f43f0400640 1 -- 172.21.15.12:0/4180998438 wait complete. 2024-04-26T21:59:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:06 smithi077 ceph-mon[32280]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:06.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:06 smithi012 ceph-mon[28184]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:06.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:06 smithi154 ceph-mon[32578]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:07.171 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:06 smithi012 podman[68306]: 2024-04-26 21:59:06.873175815 +0000 UTC m=+0.163725066 container create e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:07 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4180998438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 podman[68306]: 2024-04-26 21:59:07.260149455 +0000 UTC m=+0.550698732 container init e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 podman[68306]: 2024-04-26 21:59:07.270411301 +0000 UTC m=+0.560960569 container start e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 ceph-mds[68325]: -- 172.21.15.12:0/3333041078 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ecd2aba40 con 0x555ecd27ec00 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j[68321]: starting mds.j at 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 bash[68306]: e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea 2024-04-26T21:59:07.573 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 21:59:07 smithi012 systemd[1]: Started Ceph mds.j for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:07.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:07 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4180998438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:07.779 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:07 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4180998438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:08.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 -- 172.21.15.12:0/3166674024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e140772f0 msgr2=0x7f7e14077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3166674024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e140772f0 0x7f7e14077710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009900 tx=0x7f7e0402a410 comp rx=0 tx=0).stop 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 -- 172.21.15.12:0/3166674024 shutdown_connections 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3166674024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e14078ed0 0x7f7e14079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3166674024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078530 0x7f7e14078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3166674024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e140772f0 0x7f7e14077710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.705+0000 7f7e19c00640 1 -- 172.21.15.12:0/3166674024 >> 172.21.15.12:0/3166674024 conn(0x7f7e140fff40 msgr2=0x7f7e14102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.706+0000 7f7e19c00640 1 -- 172.21.15.12:0/3166674024 shutdown_connections 2024-04-26T21:59:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.706+0000 7f7e19c00640 1 -- 172.21.15.12:0/3166674024 wait complete. 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 Processor -- start 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 -- start start 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 0x7f7e1419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078ed0 0x7f7e141a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14114910 con 0x7f7e14078530 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14114790 con 0x7f7e14078ed0 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e19c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14114a90 con 0x7f7e140772f0 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.707+0000 7f7e13400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52322/0 (socket says 172.21.15.12:52322) 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 0x7f7e1419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 -- 172.21.15.12:0/3559519549 learned_addr learned my addr 172.21.15.12:0/3559519549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e19200640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078ed0 0x7f7e141a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 -- 172.21.15.12:0/3559519549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 msgr2=0x7f7e1419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 0x7f7e1419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 -- 172.21.15.12:0/3559519549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078ed0 msgr2=0x7f7e141a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078ed0 0x7f7e141a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 -- 172.21.15.12:0/3559519549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e141a6b70 con 0x7f7e14078530 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13e00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 0x7f7e1419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.708+0000 7f7e13400640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009d50 tx=0x7f7e0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.709+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00003b70 con 0x7f7e14078530 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.709+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e00003d10 con 0x7f7e14078530 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.709+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000108a0 con 0x7f7e14078530 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.709+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e141a6e60 con 0x7f7e14078530 2024-04-26T21:59:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.709+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141a72d0 con 0x7f7e14078530 2024-04-26T21:59:08.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.710+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e14111300 con 0x7f7e14078530 2024-04-26T21:59:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.713+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e00004000 con 0x7f7e14078530 2024-04-26T21:59:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.714+0000 7f7e10c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 0x7f7df80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.714+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7e000d0680 con 0x7f7e14078530 2024-04-26T21:59:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.714+0000 7f7e13e00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 0x7f7df80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.715+0000 7f7e13e00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 0x7f7df80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e04031500 tx=0x7f7e04005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.721+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e0009d460 con 0x7f7e14078530 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.12:6834/2930574103,v1:172.21.15.12:6835/2930574103] up:boot 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: fsmap 3 up:standby 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:08.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:08.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:08.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:08 smithi077 ceph-mon[32280]: Deploying daemon mds.d on smithi012 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.12:6834/2930574103,v1:172.21.15.12:6835/2930574103] up:boot 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: fsmap 3 up:standby 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:08.817 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:08 smithi012 ceph-mon[28184]: Deploying daemon mds.d on smithi012 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.12:6834/2930574103,v1:172.21.15.12:6835/2930574103] up:boot 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: fsmap 3 up:standby 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:08.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:08 smithi154 ceph-mon[32578]: Deploying daemon mds.d on smithi012 2024-04-26T21:59:08.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.979+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e1402dad0 con 0x7f7e14078530 2024-04-26T21:59:08.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.980+0000 7f7e10c00640 1 -- 172.21.15.12:0/3559519549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f7e0009cd60 con 0x7f7e14078530 2024-04-26T21:59:08.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:08.980 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:08.980 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 msgr2=0x7f7df80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 0x7f7df80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e04031500 tx=0x7f7e04005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 msgr2=0x7f7e1419f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009d50 tx=0x7f7e0000c6a0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 shutdown_connections 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7df80a3bb0 0x7f7df80a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e14078ed0 0x7f7e141a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e14078530 0x7f7e1419f4e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 --2- 172.21.15.12:0/3559519549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e140772f0 0x7f7e1419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:08.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 >> 172.21.15.12:0/3559519549 conn(0x7f7e140fff40 msgr2=0x7f7e14101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:08.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 shutdown_connections 2024-04-26T21:59:08.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:08.983+0000 7f7e19c00640 1 -- 172.21.15.12:0/3559519549 wait complete. 2024-04-26T21:59:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:09 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3559519549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:09.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:09 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3559519549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:09 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3559519549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:10.654 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:10 smithi012 ceph-mon[28184]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:10.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:10 smithi154 ceph-mon[32578]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:10.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:10 smithi077 ceph-mon[32280]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:11.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:11.056 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:10 smithi012 systemd[1]: Starting Ceph mds.d for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.210+0000 7f2c04800640 1 -- 172.21.15.12:0/2343492721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 msgr2=0x7f2bfc1113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.210+0000 7f2c04800640 1 --2- 172.21.15.12:0/2343492721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1113d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009ca0 tx=0x7f2bf802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 -- 172.21.15.12:0/2343492721 shutdown_connections 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 --2- 172.21.15.12:0/2343492721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1113d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 --2- 172.21.15.12:0/2343492721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc10a180 0x7f2bfc10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 --2- 172.21.15.12:0/2343492721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc108f90 0x7f2bfc1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 -- 172.21.15.12:0/2343492721 >> 172.21.15.12:0/2343492721 conn(0x7f2bfc077f40 msgr2=0x7f2bfc07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:11.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 -- 172.21.15.12:0/2343492721 shutdown_connections 2024-04-26T21:59:11.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.211+0000 7f2c04800640 1 -- 172.21.15.12:0/2343492721 wait complete. 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.212+0000 7f2c04800640 1 Processor -- start 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.212+0000 7f2c04800640 1 -- start start 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc10a180 0x7f2bfc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc114ad0 con 0x7f2bfc10ab40 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc114950 con 0x7f2bfc10a180 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c04800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc114c50 con 0x7f2bfc108f90 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c02800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c01e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c01e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:41248/0 (socket says 172.21.15.12:41248) 2024-04-26T21:59:11.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c01e00640 1 -- 172.21.15.12:0/2531540225 learned_addr learned my addr 172.21.15.12:0/2531540225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.213+0000 7f2c01400640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc10a180 0x7f2bfc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 -- 172.21.15.12:0/2531540225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc10a180 msgr2=0x7f2bfc19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc10a180 0x7f2bfc19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 -- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 msgr2=0x7f2bfc1a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 -- 172.21.15.12:0/2531540225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc1a6d70 con 0x7f2bfc108f90 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c02800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c01e00640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000e9c0 tx=0x7f2bf000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000cd80 con 0x7f2bfc108f90 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf00035f0 con 0x7f2bfc108f90 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc1a7060 con 0x7f2bfc108f90 2024-04-26T21:59:11.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0010690 con 0x7f2bfc108f90 2024-04-26T21:59:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.214+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1a7600 con 0x7f2bfc108f90 2024-04-26T21:59:11.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.216+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc1102b0 con 0x7f2bfc108f90 2024-04-26T21:59:11.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.218+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bf00037a0 con 0x7f2bfc108f90 2024-04-26T21:59:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.219+0000 7f2bf6a00640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.219+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2bf00d02e0 con 0x7f2bfc108f90 2024-04-26T21:59:11.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.219+0000 7f2c01400640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:11.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.220+0000 7f2c01400640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 0x7f2be40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc1a0660 tx=0x7f2bec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:11.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.223+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2bf009d0c0 con 0x7f2bfc108f90 2024-04-26T21:59:11.324 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 podman[68646]: 2024-04-26 21:59:10.961060246 +0000 UTC m=+0.037033261 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:11.324 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 podman[68646]: 2024-04-26 21:59:11.120369691 +0000 UTC m=+0.196342737 container create 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:59:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.479+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bfc02dad0 con 0x7f2bfc108f90 2024-04-26T21:59:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.481+0000 7f2bf6a00640 1 -- 172.21.15.12:0/2531540225 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f2bf009c9c0 con 0x7f2bfc108f90 2024-04-26T21:59:11.481 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:11.481 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-04-26T21:59:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 msgr2=0x7f2be40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 0x7f2be40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc1a0660 tx=0x7f2bec005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 msgr2=0x7f2bfc19f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000e9c0 tx=0x7f2bf000ee80 comp rx=0 tx=0).stop 2024-04-26T21:59:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 shutdown_connections 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bfc10ab40 0x7f2bfc1a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bfc10a180 0x7f2bfc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 --2- 172.21.15.12:0/2531540225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bfc108f90 0x7f2bfc19f150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 >> 172.21.15.12:0/2531540225 conn(0x7f2bfc077f40 msgr2=0x7f2bfc10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 shutdown_connections 2024-04-26T21:59:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:11.484+0000 7f2c04800640 1 -- 172.21.15.12:0/2531540225 wait complete. 2024-04-26T21:59:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:11 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2531540225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 podman[68646]: 2024-04-26 21:59:11.507091884 +0000 UTC m=+0.583064904 container init 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 podman[68646]: 2024-04-26 21:59:11.516925403 +0000 UTC m=+0.592898421 container start 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 ceph-mds[68706]: -- 172.21.15.12:0/2119900344 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fa04c7a40 con 0x560fa049ac00 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d[68700]: starting mds.d at 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 bash[68646]: 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e 2024-04-26T21:59:11.824 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 21:59:11 smithi012 systemd[1]: Started Ceph mds.d for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:11.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:11 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2531540225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:11 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2531540225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: Deploying daemon mds.e on smithi077 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.12:6836/247823794,v1:172.21.15.12:6837/247823794] up:boot 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: fsmap 4 up:standby 2024-04-26T21:59:13.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:12 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:13.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:13.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:13.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: Deploying daemon mds.e on smithi077 2024-04-26T21:59:13.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.12:6836/247823794,v1:172.21.15.12:6837/247823794] up:boot 2024-04-26T21:59:13.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: fsmap 4 up:standby 2024-04-26T21:59:13.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:12 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:59:13.224 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: Deploying daemon mds.e on smithi077 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.12:6836/247823794,v1:172.21.15.12:6837/247823794] up:boot 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: fsmap 4 up:standby 2024-04-26T21:59:13.225 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:12 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:59:13.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 -- 172.21.15.12:0/3530827725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc084109f80 msgr2=0x7fc08410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 --2- 172.21.15.12:0/3530827725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc084109f80 0x7fc08410a400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009900 tx=0x7fc06c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 -- 172.21.15.12:0/3530827725 shutdown_connections 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 --2- 172.21.15.12:0/3530827725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc08410a940 0x7fc0841111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 --2- 172.21.15.12:0/3530827725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc084109f80 0x7fc08410a400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 --2- 172.21.15.12:0/3530827725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084108d90 0x7fc0841091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 -- 172.21.15.12:0/3530827725 >> 172.21.15.12:0/3530827725 conn(0x7fc084077d30 msgr2=0x7fc08407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 -- 172.21.15.12:0/3530827725 shutdown_connections 2024-04-26T21:59:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.734+0000 7fc088e00640 1 -- 172.21.15.12:0/3530827725 wait complete. 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 Processor -- start 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 -- start start 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 0x7fc08419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 0x7fc08419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841148c0 con 0x7fc084109f80 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc084114740 con 0x7fc08410a940 2024-04-26T21:59:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.736+0000 7fc088e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc084114a40 con 0x7fc084108d90 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc07aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 0x7fc08419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:44698/0 (socket says 172.21.15.12:44698) 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 -- 172.21.15.12:0/649116747 learned_addr learned my addr 172.21.15.12:0/649116747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc07aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 0x7fc08419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45354/0 (socket says 172.21.15.12:45354) 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 0x7fc08419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 -- 172.21.15.12:0/649116747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 msgr2=0x7fc08419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 0x7fc08419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 -- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 msgr2=0x7fc08419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 0x7fc08419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 -- 172.21.15.12:0/649116747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0841a6ba0 con 0x7fc08410a940 2024-04-26T21:59:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083400640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 0x7fc08419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.737+0000 7fc083e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00eb10 tx=0x7fc07c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.738+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004b20 con 0x7fc08410a940 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.738+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c004cc0 con 0x7fc08410a940 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.738+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0841a6e90 con 0x7fc08410a940 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.738+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c010690 con 0x7fc08410a940 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.738+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0841a7350 con 0x7fc08410a940 2024-04-26T21:59:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.739+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc084110180 con 0x7fc08410a940 2024-04-26T21:59:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.741+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc07c005000 con 0x7fc08410a940 2024-04-26T21:59:13.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.743+0000 7fc080c00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 0x7fc0680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:13.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.743+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc07c0d0750 con 0x7fc08410a940 2024-04-26T21:59:13.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.743+0000 7fc083400640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 0x7fc0680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:13.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.744+0000 7fc083400640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 0x7fc0680a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc084109de0 tx=0x7fc074005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.745+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc07c054060 con 0x7fc08410a940 2024-04-26T21:59:13.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.993+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08402dd40 con 0x7fc08410a940 2024-04-26T21:59:13.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.994+0000 7fc080c00640 1 -- 172.21.15.12:0/649116747 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fc07c09d530 con 0x7fc08410a940 2024-04-26T21:59:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:13.995 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:13.995 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-04-26T21:59:13.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 msgr2=0x7fc0680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:13.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 0x7fc0680a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc084109de0 tx=0x7fc074005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:13.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 msgr2=0x7fc0841a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:13.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00eb10 tx=0x7fc07c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 shutdown_connections 2024-04-26T21:59:13.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc0680a3bb0 0x7fc0680a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc08410a940 0x7fc0841a6530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc084109f80 0x7fc08419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 --2- 172.21.15.12:0/649116747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc084108d90 0x7fc08419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:13.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 >> 172.21.15.12:0/649116747 conn(0x7fc084077d30 msgr2=0x7fc08410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:13.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 shutdown_connections 2024-04-26T21:59:13.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:13.997+0000 7fc088e00640 1 -- 172.21.15.12:0/649116747 wait complete. 2024-04-26T21:59:14.232 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 systemd[1]: Starting Ceph mds.e for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:14.573 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 podman[61638]: 2024-04-26 21:59:14.221200031 +0000 UTC m=+0.017830917 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:14.573 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 podman[61638]: 2024-04-26 21:59:14.337711347 +0000 UTC m=+0.134342235 container create fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:59:14.573 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 podman[61638]: 2024-04-26 21:59:14.57382922 +0000 UTC m=+0.370460101 container init fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T21:59:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-mon[32280]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/649116747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:14.982 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 podman[61638]: 2024-04-26 21:59:14.576378782 +0000 UTC m=+0.373009665 container start fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:59:14.983 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-mds[61657]: -- 172.21.15.77:0/1503614320 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3abfafa40 con 0x55d3abf82800 2024-04-26T21:59:14.983 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e[61653]: starting mds.e at 2024-04-26T21:59:14.983 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 bash[61638]: fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 2024-04-26T21:59:14.983 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 21:59:14 smithi077 systemd[1]: Started Ceph mds.e for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:14 smithi012 ceph-mon[28184]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/649116747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:14 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:15.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:14 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:15.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:14 smithi154 ceph-mon[32578]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/649116747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:14 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:14 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:16.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6834/3868639549,v1:172.21.15.77:6835/3868639549] up:boot 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: fsmap 5 up:standby 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:15 smithi012 ceph-mon[28184]: Deploying daemon mds.f on smithi154 2024-04-26T21:59:16.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6834/3868639549,v1:172.21.15.77:6835/3868639549] up:boot 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: fsmap 5 up:standby 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:16.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:15 smithi154 ceph-mon[32578]: Deploying daemon mds.f on smithi154 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6834/3868639549,v1:172.21.15.77:6835/3868639549] up:boot 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: fsmap 5 up:standby 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:16.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:15 smithi077 ceph-mon[32280]: Deploying daemon mds.f on smithi154 2024-04-26T21:59:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.240+0000 7f1a9b000640 1 -- 172.21.15.12:0/3335241602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a941039a0 msgr2=0x7f1a94103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.240+0000 7f1a9b000640 1 --2- 172.21.15.12:0/3335241602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a941039a0 0x7f1a94103e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009900 tx=0x7f1a8002a410 comp rx=0 tx=0).stop 2024-04-26T21:59:16.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 -- 172.21.15.12:0/3335241602 shutdown_connections 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 --2- 172.21.15.12:0/3335241602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a94104360 0x7f1a9410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 --2- 172.21.15.12:0/3335241602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a941039a0 0x7f1a94103e20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 --2- 172.21.15.12:0/3335241602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 0x7f1a94102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 -- 172.21.15.12:0/3335241602 >> 172.21.15.12:0/3335241602 conn(0x7f1a940fdd40 msgr2=0x7f1a94100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 -- 172.21.15.12:0/3335241602 shutdown_connections 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.241+0000 7f1a9b000640 1 -- 172.21.15.12:0/3335241602 wait complete. 2024-04-26T21:59:16.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.242+0000 7f1a9b000640 1 Processor -- start 2024-04-26T21:59:16.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.242+0000 7f1a9b000640 1 -- start start 2024-04-26T21:59:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 0x7f1a9419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 0x7f1a9419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:16.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 0x7f1a941a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9410e310 con 0x7f1a941039a0 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9410e190 con 0x7f1a94104360 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9b000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9410e490 con 0x7f1a941027b0 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a99200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 0x7f1a9419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a99c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 0x7f1a9419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a99200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 0x7f1a9419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45370/0 (socket says 172.21.15.12:45370) 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a99200640 1 -- 172.21.15.12:0/4173438938 learned_addr learned my addr 172.21.15.12:0/4173438938 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 0x7f1a941a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 -- 172.21.15.12:0/4173438938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 msgr2=0x7f1a9419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 0x7f1a9419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 -- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 msgr2=0x7f1a9419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 0x7f1a9419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.243+0000 7f1a9a600640 1 -- 172.21.15.12:0/4173438938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a941a27c0 con 0x7f1a94104360 2024-04-26T21:59:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a9a600640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 0x7f1a941a2150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ec50 tx=0x7f1a8c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c003b70 con 0x7f1a94104360 2024-04-26T21:59:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a8c003d10 con 0x7f1a94104360 2024-04-26T21:59:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a941a2ab0 con 0x7f1a94104360 2024-04-26T21:59:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c010830 con 0x7f1a94104360 2024-04-26T21:59:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.244+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941a2fb0 con 0x7f1a94104360 2024-04-26T21:59:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.245+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a941a32a0 con 0x7f1a94104360 2024-04-26T21:59:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.247+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1a8c003ec0 con 0x7f1a94104360 2024-04-26T21:59:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.249+0000 7f1a92000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 0x7f1a7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.249+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1a8c0d0550 con 0x7f1a94104360 2024-04-26T21:59:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.249+0000 7f1a99c00640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 0x7f1a7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.250+0000 7f1a99c00640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 0x7f1a7c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1a94103800 tx=0x7f1a88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.253+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a8c09d330 con 0x7f1a94104360 2024-04-26T21:59:16.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.501+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a941a3470 con 0x7f1a94104360 2024-04-26T21:59:16.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.502+0000 7f1a92000640 1 -- 172.21.15.12:0/4173438938 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f1a8c09cc30 con 0x7f1a94104360 2024-04-26T21:59:16.502 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:16.502 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:59:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-04-26T21:59:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 msgr2=0x7f1a7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 0x7f1a7c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1a94103800 tx=0x7f1a88005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 msgr2=0x7f1a941a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 0x7f1a941a2150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00ec50 tx=0x7f1a8c00c6a0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 shutdown_connections 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1a7c0a3bb0 0x7f1a7c0a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a94104360 0x7f1a941a2150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a941039a0 0x7f1a9419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 --2- 172.21.15.12:0/4173438938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a941027b0 0x7f1a9419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 >> 172.21.15.12:0/4173438938 conn(0x7f1a940fdd40 msgr2=0x7f1a94074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 shutdown_connections 2024-04-26T21:59:16.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:16.505+0000 7f1a9b000640 1 -- 172.21.15.12:0/4173438938 wait complete. 2024-04-26T21:59:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:16 smithi012 ceph-mon[28184]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:17.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:16 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4173438938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:17.126 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:16 smithi154 ceph-mon[32578]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:17.127 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:16 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4173438938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:17.127 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 systemd[1]: Starting Ceph mds.f for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:17.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:16 smithi077 ceph-mon[32280]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:17.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:16 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4173438938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:17.391 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 podman[62814]: 2024-04-26 21:59:17.248660484 +0000 UTC m=+0.018600851 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:17.391 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 podman[62814]: 2024-04-26 21:59:17.374840989 +0000 UTC m=+0.144781328 container create 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T21:59:17.890 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 podman[62814]: 2024-04-26 21:59:17.62812192 +0000 UTC m=+0.398062261 container init 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:59:17.891 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 podman[62814]: 2024-04-26 21:59:17.631315241 +0000 UTC m=+0.401255589 container start 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T21:59:17.891 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 ceph-mds[62833]: -- 172.21.15.154:0/397284861 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc7d819a40 con 0x55fc7d7ec800 2024-04-26T21:59:17.891 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f[62829]: starting mds.f at 2024-04-26T21:59:17.891 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 bash[62814]: 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 2024-04-26T21:59:17.891 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 21:59:17 smithi154 systemd[1]: Started Ceph mds.f for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:18.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 -- 172.21.15.12:0/4219913678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 msgr2=0x7f7e24077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 --2- 172.21.15.12:0/4219913678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e24077710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009900 tx=0x7f7e1402a410 comp rx=0 tx=0).stop 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 -- 172.21.15.12:0/4219913678 shutdown_connections 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 --2- 172.21.15.12:0/4219913678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e24079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 --2- 172.21.15.12:0/4219913678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 0x7f7e24078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:18.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 --2- 172.21.15.12:0/4219913678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e24077710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 -- 172.21.15.12:0/4219913678 >> 172.21.15.12:0/4219913678 conn(0x7f7e240fff10 msgr2=0x7f7e24102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 -- 172.21.15.12:0/4219913678 shutdown_connections 2024-04-26T21:59:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.757+0000 7f7e23200640 1 -- 172.21.15.12:0/4219913678 wait complete. 2024-04-26T21:59:18.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.758+0000 7f7e23200640 1 Processor -- start 2024-04-26T21:59:18.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.758+0000 7f7e23200640 1 -- start start 2024-04-26T21:59:18.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:18.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 0x7f7e2419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:18.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e241a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114900 con 0x7f7e24078530 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114780 con 0x7f7e24078ed0 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e23200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114a80 con 0x7f7e240772f0 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:41324/0 (socket says 172.21.15.12:41324) 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 -- 172.21.15.12:0/24009830 learned_addr learned my addr 172.21.15.12:0/24009830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21400640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 0x7f7e2419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e22800640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e241a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 -- 172.21.15.12:0/24009830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 msgr2=0x7f7e241a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e241a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 -- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 msgr2=0x7f7e2419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 0x7f7e2419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e21e00640 1 -- 172.21.15.12:0/24009830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241a6b50 con 0x7f7e240772f0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.759+0000 7f7e22800640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e241a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e21e00640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005ab0 tx=0x7f7e14003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1403f070 con 0x7f7e240772f0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e14031c50 con 0x7f7e240772f0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e241a6de0 con 0x7f7e240772f0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14035030 con 0x7f7e240772f0 2024-04-26T21:59:18.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.760+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a7240 con 0x7f7e240772f0 2024-04-26T21:59:18.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.761+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e241112f0 con 0x7f7e240772f0 2024-04-26T21:59:18.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.763+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e14042070 con 0x7f7e240772f0 2024-04-26T21:59:18.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.764+0000 7f7e1aa00640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 0x7f7e080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:18.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.765+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7e140f06e0 con 0x7f7e240772f0 2024-04-26T21:59:18.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.765+0000 7f7e21400640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 0x7f7e080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:18.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.766+0000 7f7e21400640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 0x7f7e080a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e241a0440 tx=0x7f7e10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:18.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:18.768+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e140bd4c0 con 0x7f7e240772f0 2024-04-26T21:59:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:18.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.154:6834/1762309874,v1:172.21.15.154:6835/1762309874] up:boot 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: fsmap 6 up:standby 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:18.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:18 smithi077 ceph-mon[32280]: Deploying daemon mds.c on smithi154 2024-04-26T21:59:19.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.009+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e24063680 con 0x7f7e240772f0 2024-04-26T21:59:19.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.010+0000 7f7e1aa00640 1 -- 172.21.15.12:0/24009830 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f7e14031e00 con 0x7f7e240772f0 2024-04-26T21:59:19.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:19.011 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T21:59:19.011 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2024-04-26T21:59:19.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 msgr2=0x7f7e080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:19.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 0x7f7e080a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e241a0440 tx=0x7f7e10005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:19.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 msgr2=0x7f7e2419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:19.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005ab0 tx=0x7f7e14003b90 comp rx=0 tx=0).stop 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 shutdown_connections 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7e080a3c30 0x7f7e080a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e24078ed0 0x7f7e241a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24078530 0x7f7e2419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 --2- 172.21.15.12:0/24009830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e240772f0 0x7f7e2419ef30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 >> 172.21.15.12:0/24009830 conn(0x7f7e240fff10 msgr2=0x7f7e24101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.013+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 shutdown_connections 2024-04-26T21:59:19.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:19.014+0000 7f7e23200640 1 -- 172.21.15.12:0/24009830 wait complete. 2024-04-26T21:59:19.028 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.154:6834/1762309874,v1:172.21.15.154:6835/1762309874] up:boot 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: fsmap 6 up:standby 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:19.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:18 smithi012 ceph-mon[28184]: Deploying daemon mds.c on smithi154 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.154:6834/1762309874,v1:172.21.15.154:6835/1762309874] up:boot 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: fsmap 6 up:standby 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:19.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:18 smithi154 ceph-mon[32578]: Deploying daemon mds.c on smithi154 2024-04-26T21:59:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:19 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/24009830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:19 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/24009830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:20.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:19 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/24009830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:20.314 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 systemd[1]: Starting Ceph mds.c for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:20.593 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 podman[63117]: 2024-04-26 21:59:20.217412815 +0000 UTC m=+0.018829138 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:20.593 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 podman[63117]: 2024-04-26 21:59:20.348281511 +0000 UTC m=+0.149697833 container create 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T21:59:20.593 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 podman[63117]: 2024-04-26 21:59:20.59352019 +0000 UTC m=+0.394936516 container init 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:59:20.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:20 smithi154 ceph-mon[32578]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:20 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 podman[63117]: 2024-04-26 21:59:20.596132391 +0000 UTC m=+0.397548715 container start 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 ceph-mds[63136]: -- 172.21.15.154:0/3798318531 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632173e9a40 con 0x5632173bcc00 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c[63132]: starting mds.c at 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 bash[63117]: 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 2024-04-26T21:59:20.891 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 21:59:20 smithi154 systemd[1]: Started Ceph mds.c for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:21.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:20 smithi012 ceph-mon[28184]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:20 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:20 smithi077 ceph-mon[32280]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:21.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:20 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.253+0000 7f783ce00640 1 -- 172.21.15.12:0/3102554491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7838108650 msgr2=0x7f783810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.253+0000 7f783ce00640 1 --2- 172.21.15.12:0/3102554491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7838108650 0x7f783810aaa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7830009ca0 tx=0x7f783002a8b0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 -- 172.21.15.12:0/3102554491 shutdown_connections 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 --2- 172.21.15.12:0/3102554491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7838108650 0x7f783810aaa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 --2- 172.21.15.12:0/3102554491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838105d20 0x7f7838108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 --2- 172.21.15.12:0/3102554491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78381033f0 0x7f78381057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 -- 172.21.15.12:0/3102554491 >> 172.21.15.12:0/3102554491 conn(0x7f78380fcde0 msgr2=0x7f78380ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 -- 172.21.15.12:0/3102554491 shutdown_connections 2024-04-26T21:59:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.254+0000 7f783ce00640 1 -- 172.21.15.12:0/3102554491 wait complete. 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.255+0000 7f783ce00640 1 Processor -- start 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.255+0000 7f783ce00640 1 -- start start 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.255+0000 7f783ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.255+0000 7f783ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838105d20 0x7f783819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f783ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838108650 0x7f78381a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f783ce00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f783810e4f0 con 0x7f7838108650 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f783ce00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f783810e370 con 0x7f78381033f0 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f783ce00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f783810e670 con 0x7f7838105d20 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:35524/0 (socket says 172.21.15.12:35524) 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838105d20 0x7f783819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 -- 172.21.15.12:0/3352031909 learned_addr learned my addr 172.21.15.12:0/3352031909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7836800640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838108650 0x7f78381a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 -- 172.21.15.12:0/3352031909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838105d20 msgr2=0x7f783819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838105d20 0x7f783819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 -- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838108650 msgr2=0x7f78381a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838108650 0x7f78381a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 -- 172.21.15.12:0/3352031909 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78381a29b0 con 0x7f78381033f0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.256+0000 7f7835e00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f782800d950 tx=0x7f782800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.257+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828017070 con 0x7f78381033f0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.257+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7828009c30 con 0x7f78381033f0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.257+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78381a2ca0 con 0x7f78381033f0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.257+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782800f620 con 0x7f78381033f0 2024-04-26T21:59:21.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.257+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381a3190 con 0x7f78381033f0 2024-04-26T21:59:21.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.258+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783810a140 con 0x7f78381033f0 2024-04-26T21:59:21.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.260+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7828010460 con 0x7f78381033f0 2024-04-26T21:59:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.261+0000 7f782ea00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 0x7f781c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.261+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f78280cfc60 con 0x7f78381033f0 2024-04-26T21:59:21.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.262+0000 7f7835400640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 0x7f781c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.262+0000 7f7835400640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 0x7f781c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7824005d00 tx=0x7f782400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.266+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f782809ca40 con 0x7f78381033f0 2024-04-26T21:59:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.518+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f783802dad0 con 0x7f78381033f0 2024-04-26T21:59:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.519+0000 7f782ea00640 1 -- 172.21.15.12:0/3352031909 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f7828096e20 con 0x7f78381033f0 2024-04-26T21:59:21.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:21.520 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T21:59:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 msgr2=0x7f781c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 0x7f781c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7824005d00 tx=0x7f782400b410 comp rx=0 tx=0).stop 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 msgr2=0x7f783819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f782800d950 tx=0x7f782800de10 comp rx=0 tx=0).stop 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 shutdown_connections 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f781c0a3c30 0x7f781c0a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838108650 0x7f78381a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838105d20 0x7f783819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 --2- 172.21.15.12:0/3352031909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78381033f0 0x7f783819ada0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 >> 172.21.15.12:0/3352031909 conn(0x7f78380fcde0 msgr2=0x7f7838102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 shutdown_connections 2024-04-26T21:59:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:21.522+0000 7f783ce00640 1 -- 172.21.15.12:0/3352031909 wait complete. 2024-04-26T21:59:22.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.154:6836/175106835,v1:172.21.15.154:6837/175106835] up:boot 2024-04-26T21:59:22.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: fsmap 7 up:standby 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: Deploying daemon mds.h on smithi077 2024-04-26T21:59:22.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:21 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3352031909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:22.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.154:6836/175106835,v1:172.21.15.154:6837/175106835] up:boot 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: fsmap 7 up:standby 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: Deploying daemon mds.h on smithi077 2024-04-26T21:59:22.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:21 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3352031909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.154:6836/175106835,v1:172.21.15.154:6837/175106835] up:boot 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: fsmap 7 up:standby 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:22.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:22.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:22.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: Deploying daemon mds.h on smithi077 2024-04-26T21:59:22.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:21 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3352031909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:23.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:22 smithi154 ceph-mon[32578]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:23.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:22 smithi077 ceph-mon[32280]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:23.233 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 systemd[1]: Starting Ceph mds.h for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:23.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:22 smithi012 ceph-mon[28184]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:23.529 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 podman[61950]: 2024-04-26 21:59:23.149274307 +0000 UTC m=+0.017751845 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:23.529 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 podman[61950]: 2024-04-26 21:59:23.276469021 +0000 UTC m=+0.144946543 container create 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T21:59:23.529 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 podman[61950]: 2024-04-26 21:59:23.529655197 +0000 UTC m=+0.398132720 container init 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:23.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 -- 172.21.15.12:0/2187872837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb694075310 msgr2=0x7fb694075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 --2- 172.21.15.12:0/2187872837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb694075310 0x7fb694075770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009900 tx=0x7fb67c02a480 comp rx=0 tx=0).stop 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 -- 172.21.15.12:0/2187872837 shutdown_connections 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 --2- 172.21.15.12:0/2187872837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb69410a660 0x7fb69410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 --2- 172.21.15.12:0/2187872837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb694075310 0x7fb694075770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 --2- 172.21.15.12:0/2187872837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6940768f0 0x7fb694074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 -- 172.21.15.12:0/2187872837 >> 172.21.15.12:0/2187872837 conn(0x7fb6940fdd40 msgr2=0x7fb694100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 -- 172.21.15.12:0/2187872837 shutdown_connections 2024-04-26T21:59:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.764+0000 7fb698a00640 1 -- 172.21.15.12:0/2187872837 wait complete. 2024-04-26T21:59:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.765+0000 7fb698a00640 1 Processor -- start 2024-04-26T21:59:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 -- start start 2024-04-26T21:59:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6940768f0 0x7fb694106040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb69410a660 0x7fb694102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb694110510 con 0x7fb6940768f0 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb694110390 con 0x7fb694075310 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb698a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb694110690 con 0x7fb69410a660 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb692800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb692800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:35546/0 (socket says 172.21.15.12:35546) 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb692800640 1 -- 172.21.15.12:0/3032108990 learned_addr learned my addr 172.21.15.12:0/3032108990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb693200640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb69410a660 0x7fb694102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.766+0000 7fb68be00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6940768f0 0x7fb694106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 -- 172.21.15.12:0/3032108990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb69410a660 msgr2=0x7fb694102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb69410a660 0x7fb694102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 -- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6940768f0 msgr2=0x7fb694106040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6940768f0 0x7fb694106040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 -- 172.21.15.12:0/3032108990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb694102f30 con 0x7fb694075310 2024-04-26T21:59:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb692800640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb68400ea60 tx=0x7fb68400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68400cde0 con 0x7fb694075310 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.767+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941a7010 con 0x7fb694075310 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.768+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6840035f0 con 0x7fb694075310 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.768+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684010690 con 0x7fb694075310 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.768+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6941a7410 con 0x7fb694075310 2024-04-26T21:59:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.769+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694069670 con 0x7fb694075310 2024-04-26T21:59:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.771+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6840037a0 con 0x7fb694075310 2024-04-26T21:59:23.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.773+0000 7fb68b400640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 0x7fb6780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.773+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6840d0290 con 0x7fb694075310 2024-04-26T21:59:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.773+0000 7fb68be00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 0x7fb6780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.774+0000 7fb68be00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 0x7fb6780a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb67c031500 tx=0x7fb67c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:23.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:23.777+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb68409d070 con 0x7fb694075310 2024-04-26T21:59:23.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 podman[61950]: 2024-04-26 21:59:23.533195178 +0000 UTC m=+0.401672702 container start 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T21:59:23.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 ceph-mds[61969]: -- 172.21.15.77:0/510637343 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc334dda40 con 0x55dc334b0c00 2024-04-26T21:59:23.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h[61965]: starting mds.h at 2024-04-26T21:59:23.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 bash[61950]: 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f 2024-04-26T21:59:23.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 21:59:23 smithi077 systemd[1]: Started Ceph mds.h for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:24.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.027+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb694103100 con 0x7fb694075310 2024-04-26T21:59:24.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.029+0000 7fb68b400640 1 -- 172.21.15.12:0/3032108990 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7fb68409c970 con 0x7fb694075310 2024-04-26T21:59:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:24.029 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T21:59:24.029 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 msgr2=0x7fb6780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 0x7fb6780a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb67c031500 tx=0x7fb67c005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 msgr2=0x7fb694105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb68400ea60 tx=0x7fb68400ef20 comp rx=0 tx=0).stop 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 shutdown_connections 2024-04-26T21:59:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb6780a3bb0 0x7fb6780a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb69410a660 0x7fb694102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6940768f0 0x7fb694106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 --2- 172.21.15.12:0/3032108990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb694075310 0x7fb694105b00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.031+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 >> 172.21.15.12:0/3032108990 conn(0x7fb6940fdd40 msgr2=0x7fb6941065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.032+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 shutdown_connections 2024-04-26T21:59:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:24.032+0000 7fb698a00640 1 -- 172.21.15.12:0/3032108990 wait complete. 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: Deploying daemon mds.i on smithi154 2024-04-26T21:59:24.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6836/1703155205,v1:172.21.15.77:6837/1703155205] up:boot 2024-04-26T21:59:24.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: fsmap 8 up:standby 2024-04-26T21:59:24.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:24.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3032108990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: Deploying daemon mds.i on smithi154 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6836/1703155205,v1:172.21.15.77:6837/1703155205] up:boot 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: fsmap 8 up:standby 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:24.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3032108990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: Deploying daemon mds.i on smithi154 2024-04-26T21:59:25.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6836/1703155205,v1:172.21.15.77:6837/1703155205] up:boot 2024-04-26T21:59:25.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: fsmap 8 up:standby 2024-04-26T21:59:25.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:25.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3032108990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:26.169 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:25 smithi154 systemd[1]: Starting Ceph mds.i for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.389+0000 7fdf74800640 1 -- 172.21.15.12:0/3458146989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c104520 msgr2=0x7fdf6c10adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.389+0000 7fdf74800640 1 --2- 172.21.15.12:0/3458146989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c104520 0x7fdf6c10adb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009ca0 tx=0x7fdf6802a8b0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 -- 172.21.15.12:0/3458146989 shutdown_connections 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 --2- 172.21.15.12:0/3458146989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c104520 0x7fdf6c10adb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 --2- 172.21.15.12:0/3458146989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 0x7fdf6c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 --2- 172.21.15.12:0/3458146989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c102970 0x7fdf6c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 -- 172.21.15.12:0/3458146989 >> 172.21.15.12:0/3458146989 conn(0x7fdf6c0fdee0 msgr2=0x7fdf6c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 -- 172.21.15.12:0/3458146989 shutdown_connections 2024-04-26T21:59:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.390+0000 7fdf74800640 1 -- 172.21.15.12:0/3458146989 wait complete. 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.391+0000 7fdf74800640 1 Processor -- start 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.391+0000 7fdf74800640 1 -- start start 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 0x7fdf6c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 0x7fdf6c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e4d0 con 0x7fdf6c102970 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e350 con 0x7fdf6c104520 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf74800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c10e650 con 0x7fdf6c103b60 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf71e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf71e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34908/0 (socket says 172.21.15.12:34908) 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf71e00640 1 -- 172.21.15.12:0/617395306 learned_addr learned my addr 172.21.15.12:0/617395306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 0x7fdf6c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf71400640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 0x7fdf6c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 -- 172.21.15.12:0/617395306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 msgr2=0x7fdf6c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 0x7fdf6c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 -- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 msgr2=0x7fdf6c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf72800640 1 -- 172.21.15.12:0/617395306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c1a07a0 con 0x7fdf6c104520 2024-04-26T21:59:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.392+0000 7fdf71e00640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf72800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 0x7fdf6c1a0130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf680307a0 tx=0x7fdf68003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68004240 con 0x7fdf6c104520 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf68040070 con 0x7fdf6c104520 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c074b60 con 0x7fdf6c104520 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68031ce0 con 0x7fdf6c104520 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.393+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c0750b0 con 0x7fdf6c104520 2024-04-26T21:59:26.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.394+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c109c30 con 0x7fdf6c104520 2024-04-26T21:59:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.396+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf68031440 con 0x7fdf6c104520 2024-04-26T21:59:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.398+0000 7fdf66a00640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 0x7fdf540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.398+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf680f1160 con 0x7fdf6c104520 2024-04-26T21:59:26.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.398+0000 7fdf71e00640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 0x7fdf540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.399+0000 7fdf71e00640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 0x7fdf540a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf600104e0 tx=0x7fdf6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.402+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf680bdf40 con 0x7fdf6c104520 2024-04-26T21:59:26.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 podman[63421]: 2024-04-26 21:59:26.060296635 +0000 UTC m=+0.017441468 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:26.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 podman[63421]: 2024-04-26 21:59:26.184185762 +0000 UTC m=+0.141330593 container create 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T21:59:26.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 podman[63421]: 2024-04-26 21:59:26.443769084 +0000 UTC m=+0.400913915 container init 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:59:26.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.658+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf6c076490 con 0x7fdf6c104520 2024-04-26T21:59:26.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.662+0000 7fdf66a00640 1 -- 172.21.15.12:0/617395306 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7fdf680bd840 con 0x7fdf6c104520 2024-04-26T21:59:26.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:26.663 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T21:59:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-04-26T21:59:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 msgr2=0x7fdf540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 0x7fdf540a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf600104e0 tx=0x7fdf6000b410 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 msgr2=0x7fdf6c1a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 0x7fdf6c1a0130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf680307a0 tx=0x7fdf68003ba0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 shutdown_connections 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf540a3bb0 0x7fdf540a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c104520 0x7fdf6c1a0130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c103b60 0x7fdf6c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 --2- 172.21.15.12:0/617395306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c102970 0x7fdf6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.665+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 >> 172.21.15.12:0/617395306 conn(0x7fdf6c0fdee0 msgr2=0x7fdf6c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.666+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 shutdown_connections 2024-04-26T21:59:26.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:26.666+0000 7fdf74800640 1 -- 172.21.15.12:0/617395306 wait complete. 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 podman[63421]: 2024-04-26 21:59:26.446904695 +0000 UTC m=+0.404049528 container start 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mds[63440]: -- 172.21.15.154:0/2411050428 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d64dafa40 con 0x559d64d82800 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i[63436]: starting mds.i at 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 bash[63421]: 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 21:59:26 smithi154 systemd[1]: Started Ceph mds.i for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/617395306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:26.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:26.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:26 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/617395306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:26.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:26 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/617395306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:27.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:26 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:27.862 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:27 smithi154 ceph-mon[32578]: Deploying daemon mds.g on smithi012 2024-04-26T21:59:27.863 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:27 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.154:6838/404103825,v1:172.21.15.154:6839/404103825] up:boot 2024-04-26T21:59:27.863 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:27 smithi154 ceph-mon[32578]: fsmap 9 up:standby 2024-04-26T21:59:27.863 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:27.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:27 smithi012 ceph-mon[28184]: Deploying daemon mds.g on smithi012 2024-04-26T21:59:27.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:27 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.154:6838/404103825,v1:172.21.15.154:6839/404103825] up:boot 2024-04-26T21:59:27.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:27 smithi012 ceph-mon[28184]: fsmap 9 up:standby 2024-04-26T21:59:27.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:27 smithi077 ceph-mon[32280]: Deploying daemon mds.g on smithi012 2024-04-26T21:59:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:27 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.154:6838/404103825,v1:172.21.15.154:6839/404103825] up:boot 2024-04-26T21:59:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:27 smithi077 ceph-mon[32280]: fsmap 9 up:standby 2024-04-26T21:59:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:28.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:28.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:28 smithi012 ceph-mon[28184]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:28.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 -- 172.21.15.12:0/418735728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec100410 msgr2=0x7fe8ec100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/418735728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec100410 0x7fe8ec100890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009900 tx=0x7fe8d802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 -- 172.21.15.12:0/418735728 shutdown_connections 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/418735728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 0x7fe8ec1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/418735728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec100410 0x7fe8ec100890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/418735728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec101940 0x7fe8ec101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 -- 172.21.15.12:0/418735728 >> 172.21.15.12:0/418735728 conn(0x7fe8ec0fbef0 msgr2=0x7fe8ec0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.911+0000 7fe8ebe00640 1 -- 172.21.15.12:0/418735728 shutdown_connections 2024-04-26T21:59:28.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.912+0000 7fe8ebe00640 1 -- 172.21.15.12:0/418735728 wait complete. 2024-04-26T21:59:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.913+0000 7fe8ebe00640 1 Processor -- start 2024-04-26T21:59:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.913+0000 7fe8ebe00640 1 -- start start 2024-04-26T21:59:28.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:28.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 0x7fe8ec19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:28.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 0x7fe8ec1a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:28.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec1104f0 con 0x7fe8ec100410 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec110370 con 0x7fe8ec100dd0 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ebe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec110670 con 0x7fe8ec101940 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 0x7fe8ec1a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33406/0 (socket says 172.21.15.12:33406) 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 -- 172.21.15.12:0/4014857898 learned_addr learned my addr 172.21.15.12:0/4014857898 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8ea000640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 0x7fe8ec19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 -- 172.21.15.12:0/4014857898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 msgr2=0x7fe8ec1a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 0x7fe8ec1a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 -- 172.21.15.12:0/4014857898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 msgr2=0x7fe8ec19b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 0x7fe8ec19b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eaa00640 1 -- 172.21.15.12:0/4014857898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec1a27c0 con 0x7fe8ec100410 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.914+0000 7fe8eb400640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 0x7fe8ec1a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8eaa00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e9f0 tx=0x7fe8dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00cd80 con 0x7fe8ec100410 2024-04-26T21:59:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec1a2ab0 con 0x7fe8ec100410 2024-04-26T21:59:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc0108e0 con 0x7fe8ec100410 2024-04-26T21:59:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec1a3020 con 0x7fe8ec100410 2024-04-26T21:59:28.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.915+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc018920 con 0x7fe8ec100410 2024-04-26T21:59:28.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.916+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec10cc10 con 0x7fe8ec100410 2024-04-26T21:59:28.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.919+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe8dc010430 con 0x7fe8ec100410 2024-04-26T21:59:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.920+0000 7fe8e3400640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 0x7fe8d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.920+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8dc0d04d0 con 0x7fe8ec100410 2024-04-26T21:59:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.920+0000 7fe8ea000640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 0x7fe8d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.921+0000 7fe8ea000640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 0x7fe8d00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005ab0 tx=0x7fe8d8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:28.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:28.923+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8dc09cc60 con 0x7fe8ec100410 2024-04-26T21:59:28.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:28 smithi077 ceph-mon[32280]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:29.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:28 smithi154 ceph-mon[32578]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:29.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.175+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8ec02dc90 con 0x7fe8ec100410 2024-04-26T21:59:29.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.176+0000 7fe8e3400640 1 -- 172.21.15.12:0/4014857898 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7995 (secure 0 0 0) 0x7fe8dc0a0f60 con 0x7fe8ec100410 2024-04-26T21:59:29.176 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:29.176 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T21:59:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-04-26T21:59:29.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.178+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 msgr2=0x7fe8d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.178+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 0x7fe8d00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005ab0 tx=0x7fe8d8005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:29.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.178+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 msgr2=0x7fe8ec19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.178+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e9f0 tx=0x7fe8dc00eeb0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 shutdown_connections 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8d00a3bb0 0x7fe8d00a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8ec101940 0x7fe8ec1a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ec100dd0 0x7fe8ec19b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 --2- 172.21.15.12:0/4014857898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8ec100410 0x7fe8ec19aba0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 >> 172.21.15.12:0/4014857898 conn(0x7fe8ec0fbef0 msgr2=0x7fe8ec0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 shutdown_connections 2024-04-26T21:59:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:29.179+0000 7fe8ebe00640 1 -- 172.21.15.12:0/4014857898 wait complete. 2024-04-26T21:59:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:29 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4014857898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:29 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4014857898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:30.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:29 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4014857898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:30.230 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:29 smithi012 systemd[1]: Starting Ceph mds.g for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:30.539 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 podman[69294]: 2024-04-26 21:59:30.136329402 +0000 UTC m=+0.040945106 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:30.539 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 podman[69294]: 2024-04-26 21:59:30.267304863 +0000 UTC m=+0.171920592 container create 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:59:30.539 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 podman[69294]: 2024-04-26 21:59:30.537929627 +0000 UTC m=+0.442545362 container init 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T21:59:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:30 smithi012 ceph-mon[28184]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:30.823 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 podman[69294]: 2024-04-26 21:59:30.551638973 +0000 UTC m=+0.456254686 container start 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T21:59:30.824 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 ceph-mds[69314]: -- 172.21.15.12:0/963821584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620dbea5a40 con 0x5620dbe79000 2024-04-26T21:59:30.824 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: starting mds.g at 2024-04-26T21:59:30.824 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 bash[69294]: 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 2024-04-26T21:59:30.824 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 21:59:30 smithi012 systemd[1]: Started Ceph mds.g for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:30.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:30 smithi077 ceph-mon[32280]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:31.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:30 smithi154 ceph-mon[32578]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:31.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:31.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 -- 172.21.15.12:0/3027034775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c076970 msgr2=0x7f977c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 --2- 172.21.15.12:0/3027034775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c076970 0x7f977c074dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f976c009930 tx=0x7f976c02a420 comp rx=0 tx=0).stop 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 -- 172.21.15.12:0/3027034775 shutdown_connections 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 --2- 172.21.15.12:0/3027034775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c10ed60 0x7f977c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 --2- 172.21.15.12:0/3027034775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c075390 0x7f977c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 --2- 172.21.15.12:0/3027034775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c076970 0x7f977c074dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 -- 172.21.15.12:0/3027034775 >> 172.21.15.12:0/3027034775 conn(0x7f977c0fdd00 msgr2=0x7f977c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 -- 172.21.15.12:0/3027034775 shutdown_connections 2024-04-26T21:59:31.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.396+0000 7f9781200640 1 -- 172.21.15.12:0/3027034775 wait complete. 2024-04-26T21:59:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.397+0000 7f9781200640 1 Processor -- start 2024-04-26T21:59:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 -- start start 2024-04-26T21:59:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 0x7f977c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:31.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 0x7f977c1a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f977c114900 con 0x7f977c075390 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f977c114780 con 0x7f977c10ed60 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f9781200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f977c114a80 con 0x7f977c076970 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 0x7f977c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 0x7f977c1a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50034/0 (socket says 172.21.15.12:50034) 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977aa00640 1 -- 172.21.15.12:0/794053035 learned_addr learned my addr 172.21.15.12:0/794053035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.398+0000 7f977b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 0x7f977c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33450/0 (socket says 172.21.15.12:33450) 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 -- 172.21.15.12:0/794053035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 msgr2=0x7f977c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 -- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 msgr2=0x7f977c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 0x7f977c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 -- 172.21.15.12:0/794053035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a6be0 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977aa00640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f977be00640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 0x7f977c1a6570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f977400e940 tx=0x7f977400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774004d20 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9774005020 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.399+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a6ed0 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.400+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97740116d0 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.400+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a7390 con 0x7f977c10ed60 2024-04-26T21:59:31.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.401+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c104f30 con 0x7f977c10ed60 2024-04-26T21:59:31.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.405+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9774005460 con 0x7f977c10ed60 2024-04-26T21:59:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.406+0000 7f9773e00640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 0x7f97600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.406+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97740d07e0 con 0x7f977c10ed60 2024-04-26T21:59:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.406+0000 7f977b400640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 0x7f97600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:31.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.407+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9774016030 con 0x7f977c10ed60 2024-04-26T21:59:31.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.407+0000 7f977b400640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 0x7f97600a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f976c005ab0 tx=0x7f976c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:31.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.708+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f977c02dad0 con 0x7f977c10ed60 2024-04-26T21:59:31.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.709+0000 7f9773e00640 1 -- 172.21.15.12:0/794053035 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f977409cf70 con 0x7f977c10ed60 2024-04-26T21:59:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:31.710 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T21:59:31.710 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-04-26T21:59:31.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 msgr2=0x7f97600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 0x7f97600a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f976c005ab0 tx=0x7f976c005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 msgr2=0x7f977c1a6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 0x7f977c1a6570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f977400e940 tx=0x7f977400ee00 comp rx=0 tx=0).stop 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 shutdown_connections 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f97600a3bb0 0x7f97600a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f977c10ed60 0x7f977c1a6570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f977c076970 0x7f977c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 --2- 172.21.15.12:0/794053035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f977c075390 0x7f977c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 >> 172.21.15.12:0/794053035 conn(0x7f977c0fdd00 msgr2=0x7f977c102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 shutdown_connections 2024-04-26T21:59:31.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:31.712+0000 7f9781200640 1 -- 172.21.15.12:0/794053035 wait complete. 2024-04-26T21:59:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.12:6838/3734088968,v1:172.21.15.12:6839/3734088968] up:boot 2024-04-26T21:59:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: fsmap 10 up:standby 2024-04-26T21:59:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: Deploying daemon mds.a on smithi012 2024-04-26T21:59:31.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:31 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/794053035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.12:6838/3734088968,v1:172.21.15.12:6839/3734088968] up:boot 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: fsmap 10 up:standby 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:32.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:32.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:32.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: Deploying daemon mds.a on smithi012 2024-04-26T21:59:32.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:31 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/794053035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:32.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.12:6838/3734088968,v1:172.21.15.12:6839/3734088968] up:boot 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: fsmap 10 up:standby 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: Deploying daemon mds.a on smithi012 2024-04-26T21:59:32.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:31 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/794053035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:32.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:32 smithi012 ceph-mon[28184]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:33.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:32 smithi154 ceph-mon[32578]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:32 smithi077 ceph-mon[32280]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:33.733 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 -- 172.21.15.12:0/123661409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0104320 msgr2=0x7fe7d010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 --2- 172.21.15.12:0/123661409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0104320 0x7fe7d010abb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009be0 tx=0x7fe7c802a6d0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 -- 172.21.15.12:0/123661409 shutdown_connections 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 --2- 172.21.15.12:0/123661409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0104320 0x7fe7d010abb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 --2- 172.21.15.12:0/123661409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0103960 0x7fe7d0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 --2- 172.21.15.12:0/123661409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0102770 0x7fe7d0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 -- 172.21.15.12:0/123661409 >> 172.21.15.12:0/123661409 conn(0x7fe7d00fdcc0 msgr2=0x7fe7d0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 -- 172.21.15.12:0/123661409 shutdown_connections 2024-04-26T21:59:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.072+0000 7fe7cf200640 1 -- 172.21.15.12:0/123661409 wait complete. 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 Processor -- start 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 -- start start 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 0x7fe7d019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0104320 0x7fe7d01a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e280 con 0x7fe7d0102770 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e100 con 0x7fe7d0103960 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.073+0000 7fe7cf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d010e400 con 0x7fe7d0104320 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 0x7fe7d019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33460/0 (socket says 172.21.15.12:33460) 2024-04-26T21:59:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cde00640 1 -- 172.21.15.12:0/2078050347 learned_addr learned my addr 172.21.15.12:0/2078050347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7ce800640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0104320 0x7fe7d01a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 -- 172.21.15.12:0/2078050347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0104320 msgr2=0x7fe7d01a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0104320 0x7fe7d01a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 -- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 msgr2=0x7fe7d019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 -- 172.21.15.12:0/2078050347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01a26a0 con 0x7fe7d0103960 2024-04-26T21:59:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cde00640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.074+0000 7fe7cd400640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 0x7fe7d019afc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.075+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00cd00 con 0x7fe7d0103960 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.075+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7bc0035f0 con 0x7fe7d0103960 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.075+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc010690 con 0x7fe7d0103960 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.075+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a2990 con 0x7fe7d0103960 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.075+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a2e90 con 0x7fe7d0103960 2024-04-26T21:59:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.076+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d01a3120 con 0x7fe7d0103960 2024-04-26T21:59:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.078+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe7bc003aa0 con 0x7fe7d0103960 2024-04-26T21:59:34.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.080+0000 7fe7c6a00640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 0x7fe7b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.080+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe7bc0d0360 con 0x7fe7d0103960 2024-04-26T21:59:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.080+0000 7fe7cde00640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 0x7fe7b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.081+0000 7fe7cde00640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 0x7fe7b40a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000f4d0 tx=0x7fe7c000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.082+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7bc09d140 con 0x7fe7d0103960 2024-04-26T21:59:34.323 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:33 smithi012 systemd[1]: Starting Ceph mds.a for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.326+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d01a3460 con 0x7fe7d0103960 2024-04-26T21:59:34.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.328+0000 7fe7c6a00640 1 -- 172.21.15.12:0/2078050347 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7fe7bc09ca40 con 0x7fe7d0103960 2024-04-26T21:59:34.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:34.328 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T21:59:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-04-26T21:59:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 msgr2=0x7fe7b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 0x7fe7b40a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000f4d0 tx=0x7fe7c000a3b0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 msgr2=0x7fe7d019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 0x7fe7d019afc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00ee50 comp rx=0 tx=0).stop 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 shutdown_connections 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe7b40a3c30 0x7fe7b40a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7d0104320 0x7fe7d01a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7d0103960 0x7fe7d019afc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 --2- 172.21.15.12:0/2078050347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7d0102770 0x7fe7d019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 >> 172.21.15.12:0/2078050347 conn(0x7fe7d00fdcc0 msgr2=0x7fe7d0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.330+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 shutdown_connections 2024-04-26T21:59:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:34.331+0000 7fe7cf200640 1 -- 172.21.15.12:0/2078050347 wait complete. 2024-04-26T21:59:34.603 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 podman[69679]: 2024-04-26 21:59:34.245353967 +0000 UTC m=+0.037189071 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:34.603 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 podman[69679]: 2024-04-26 21:59:34.365491646 +0000 UTC m=+0.157326782 container create 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:59:34.603 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 podman[69679]: 2024-04-26 21:59:34.602681144 +0000 UTC m=+0.394516304 container init 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:35.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-mon[28184]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:35.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2078050347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 podman[69679]: 2024-04-26 21:59:34.61490808 +0000 UTC m=+0.406743191 container start 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-mds[69700]: -- 172.21.15.12:0/2020103847 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654d0601a40 con 0x5654d05d4c00 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a[69696]: starting mds.a at 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 bash[69679]: 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 2024-04-26T21:59:35.074 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 21:59:34 smithi012 systemd[1]: Started Ceph mds.a for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:34 smithi154 ceph-mon[32578]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:34 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2078050347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:34 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:35.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:34 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:34 smithi077 ceph-mon[32280]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:34 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2078050347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:34 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:34 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.12:6840/1291467894,v1:172.21.15.12:6841/1291467894] up:boot 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: fsmap 11 up:standby 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:35 smithi154 ceph-mon[32578]: Deploying daemon mds.b on smithi077 2024-04-26T21:59:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.12:6840/1291467894,v1:172.21.15.12:6841/1291467894] up:boot 2024-04-26T21:59:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: fsmap 11 up:standby 2024-04-26T21:59:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:35 smithi077 ceph-mon[32280]: Deploying daemon mds.b on smithi077 2024-04-26T21:59:36.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.12:6840/1291467894,v1:172.21.15.12:6841/1291467894] up:boot 2024-04-26T21:59:36.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: fsmap 11 up:standby 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:36.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:35 smithi012 ceph-mon[28184]: Deploying daemon mds.b on smithi077 2024-04-26T21:59:36.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:36.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.577+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1646316363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28102970 msgr2=0x7f6a28102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.577+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1646316363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28102970 0x7f6a28102d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009900 tx=0x7f6a1002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1646316363 shutdown_connections 2024-04-26T21:59:36.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1646316363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28104520 0x7f6a2810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1646316363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28103b60 0x7f6a28103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1646316363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28102970 0x7f6a28102d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1646316363 >> 172.21.15.12:0/1646316363 conn(0x7f6a280fdee0 msgr2=0x7f6a28100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1646316363 shutdown_connections 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.578+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1646316363 wait complete. 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 Processor -- start 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 -- start start 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28103b60 0x7f6a28074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 0x7f6a28075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810e4a0 con 0x7f6a28104520 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810e320 con 0x7f6a28102970 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.579+0000 7f6a2ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810e620 con 0x7f6a28103b60 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a2ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 0x7f6a28075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52190/0 (socket says 172.21.15.12:52190) 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 -- 172.21.15.12:0/1214381500 learned_addr learned my addr 172.21.15.12:0/1214381500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27400640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28103b60 0x7f6a28074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 -- 172.21.15.12:0/1214381500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28103b60 msgr2=0x7f6a28074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28103b60 0x7f6a28074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 -- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 msgr2=0x7f6a28075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 0x7f6a28075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 -- 172.21.15.12:0/1214381500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a28075af0 con 0x7f6a28102970 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a2ca00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 0x7f6a28075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.580+0000 7f6a27e00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a10006010 tx=0x7f6a100042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.581+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1003f030 con 0x7f6a28102970 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.581+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a281a2c30 con 0x7f6a28102970 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.581+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a10031910 con 0x7f6a28102970 2024-04-26T21:59:36.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.581+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10035070 con 0x7f6a28102970 2024-04-26T21:59:36.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.581+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a31d0 con 0x7f6a28102970 2024-04-26T21:59:36.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.582+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a28109d10 con 0x7f6a28102970 2024-04-26T21:59:36.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.584+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a10042030 con 0x7f6a28102970 2024-04-26T21:59:36.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.585+0000 7f6a24c00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 0x7f6a0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.586+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6a100f06f0 con 0x7f6a28102970 2024-04-26T21:59:36.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.586+0000 7f6a27400640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 0x7f6a0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.586+0000 7f6a27400640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 0x7f6a0c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a28076450 tx=0x7f6a18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:36.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.588+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6a100bce80 con 0x7f6a28102970 2024-04-26T21:59:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.835+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2802dd40 con 0x7f6a28102970 2024-04-26T21:59:36.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.836+0000 7f6a24c00640 1 -- 172.21.15.12:0/1214381500 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7f6a100c1180 con 0x7f6a28102970 2024-04-26T21:59:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:36.837 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T21:59:36.837 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 msgr2=0x7f6a0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 0x7f6a0c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a28076450 tx=0x7f6a18005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 msgr2=0x7f6a280766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a10006010 tx=0x7f6a100042f0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 shutdown_connections 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6a0c0a3bb0 0x7f6a0c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a28104520 0x7f6a28075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a28103b60 0x7f6a28074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 --2- 172.21.15.12:0/1214381500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a28102970 0x7f6a280766e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 >> 172.21.15.12:0/1214381500 conn(0x7f6a280fdee0 msgr2=0x7f6a28106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:36.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 shutdown_connections 2024-04-26T21:59:36.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:36.839+0000 7f6a2ea00640 1 -- 172.21.15.12:0/1214381500 wait complete. 2024-04-26T21:59:37.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:36 smithi154 ceph-mon[32578]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:37.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:36 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1214381500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:36 smithi077 ceph-mon[32280]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:36 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1214381500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:37.233 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 systemd[1]: Starting Ceph mds.b for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T21:59:37.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:36 smithi012 ceph-mon[28184]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:37.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:36 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1214381500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:37.568 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 podman[62259]: 2024-04-26 21:59:37.183560663 +0000 UTC m=+0.017610839 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:59:37.568 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 podman[62259]: 2024-04-26 21:59:37.299140816 +0000 UTC m=+0.133191022 container create 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T21:59:37.568 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 podman[62259]: 2024-04-26 21:59:37.568768815 +0000 UTC m=+0.402818979 container init 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:59:37.982 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 podman[62259]: 2024-04-26 21:59:37.57150594 +0000 UTC m=+0.405556112 container start 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:59:37.983 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 ceph-mds[62278]: -- 172.21.15.77:0/2768946094 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb68c81a40 con 0x55fb68c55000 2024-04-26T21:59:37.983 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: starting mds.b at 2024-04-26T21:59:37.983 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 bash[62259]: 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c 2024-04-26T21:59:37.983 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:37 smithi077 systemd[1]: Started Ceph mds.b for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T21:59:38.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 -- 172.21.15.12:0/1936717215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 msgr2=0x7f65501057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 --2- 172.21.15.12:0/1936717215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f65501057d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 -- 172.21.15.12:0/1936717215 shutdown_connections 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 --2- 172.21.15.12:0/1936717215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550108640 0x7f655010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 --2- 172.21.15.12:0/1936717215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550105d10 0x7f6550108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 --2- 172.21.15.12:0/1936717215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f65501057d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 -- 172.21.15.12:0/1936717215 >> 172.21.15.12:0/1936717215 conn(0x7f65500fcdf0 msgr2=0x7f65500ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:39.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 -- 172.21.15.12:0/1936717215 shutdown_connections 2024-04-26T21:59:39.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.056+0000 7f6557a00640 1 -- 172.21.15.12:0/1936717215 wait complete. 2024-04-26T21:59:39.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.057+0000 7f6557a00640 1 Processor -- start 2024-04-26T21:59:39.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.057+0000 7f6557a00640 1 -- start start 2024-04-26T21:59:39.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f6550102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550105d10 0x7f6550101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f655010e490 con 0x7f65501033e0 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f655010e310 con 0x7f6550105d10 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6557a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f655010e610 con 0x7f6550108640 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6555000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f6550102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6555a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f654fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550105d10 0x7f6550101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6555a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:54256/0 (socket says 172.21.15.12:54256) 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.058+0000 7f6555a00640 1 -- 172.21.15.12:0/3714141261 learned_addr learned my addr 172.21.15.12:0/3714141261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 -- 172.21.15.12:0/3714141261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550105d10 msgr2=0x7f6550101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550105d10 0x7f6550101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 -- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 msgr2=0x7f6550102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f6550102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 -- 172.21.15.12:0/3714141261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6550102160 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f6555a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f654800eb10 tx=0x7f654800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.059+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548004b20 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.060+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a2ce0 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.060+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6548004cc0 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.060+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548010670 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.060+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a3180 con 0x7f6550108640 2024-04-26T21:59:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.061+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65500fe8e0 con 0x7f6550108640 2024-04-26T21:59:39.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.063+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6548005020 con 0x7f6550108640 2024-04-26T21:59:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.064+0000 7f654d600640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 0x7f65380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.065+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f65480d05b0 con 0x7f6550108640 2024-04-26T21:59:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.065+0000 7f6555000640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 0x7f65380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.066+0000 7f6555000640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 0x7f65380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:39.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.066+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f654809d390 con 0x7f6550108640 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:boot 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: fsmap 12 up:standby 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:38 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:boot 2024-04-26T21:59:39.080 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: fsmap 12 up:standby 2024-04-26T21:59:39.081 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:39.081 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.081 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:38 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:39.113 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:39.113 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:boot 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: fsmap 12 up:standby 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:39.114 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:38 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:39.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.349+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655002dad0 con 0x7f6550108640 2024-04-26T21:59:39.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.350+0000 7f654d600640 1 -- 172.21.15.12:0/3714141261 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f654809cc90 con 0x7f6550108640 2024-04-26T21:59:39.351 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:39.351 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:59:39.351 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-04-26T21:59:39.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 msgr2=0x7f65380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 0x7f65380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6544000c00 tx=0x7f65440314a0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 msgr2=0x7f65501018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f654800eb10 tx=0x7f654800c2d0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 shutdown_connections 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f65380a3bb0 0x7f65380a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6550108640 0x7f65501018d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6550105d10 0x7f6550101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 --2- 172.21.15.12:0/3714141261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501033e0 0x7f6550102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.353+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 >> 172.21.15.12:0/3714141261 conn(0x7f65500fcdf0 msgr2=0x7f6550103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.354+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 shutdown_connections 2024-04-26T21:59:39.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.354+0000 7f6557a00640 1 -- 172.21.15.12:0/3714141261 wait complete. 2024-04-26T21:59:39.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 -- 172.21.15.12:0/1044924023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 msgr2=0x7ff68c1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 --2- 172.21.15.12:0/1044924023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c1014a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009900 tx=0x7ff67c02a480 comp rx=0 tx=0).stop 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 -- 172.21.15.12:0/1044924023 shutdown_connections 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 --2- 172.21.15.12:0/1044924023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c102380 0x7ff68c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 --2- 172.21.15.12:0/1044924023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c1019e0 0x7ff68c101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 --2- 172.21.15.12:0/1044924023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c1014a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 -- 172.21.15.12:0/1044924023 >> 172.21.15.12:0/1044924023 conn(0x7ff68c0fcbe0 msgr2=0x7ff68c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 -- 172.21.15.12:0/1044924023 shutdown_connections 2024-04-26T21:59:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.578+0000 7ff68b200640 1 -- 172.21.15.12:0/1044924023 wait complete. 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 Processor -- start 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 -- start start 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c1019e0 0x7ff68c19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c110510 con 0x7ff68c1019e0 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c110390 con 0x7ff68c101080 2024-04-26T21:59:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.579+0000 7ff68b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c110690 con 0x7ff68c102380 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff689e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:54268/0 (socket says 172.21.15.12:54268) 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 -- 172.21.15.12:0/2674424863 learned_addr learned my addr 172.21.15.12:0/2674424863 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff689400640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c1019e0 0x7ff68c19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 -- 172.21.15.12:0/2674424863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 msgr2=0x7ff68c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 -- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c1019e0 msgr2=0x7ff68c19b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c1019e0 0x7ff68c19b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 -- 172.21.15.12:0/2674424863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c1a27b0 con 0x7ff68c102380 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff689e00640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.580+0000 7ff68a800640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff68400eb10 tx=0x7ff68400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.581+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684004b20 con 0x7ff68c102380 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.581+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c1a2aa0 con 0x7ff68c102380 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.581+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff684004cc0 con 0x7ff68c102380 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.581+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684010670 con 0x7ff68c102380 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.581+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a2ff0 con 0x7ff68c102380 2024-04-26T21:59:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.583+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c10bdd0 con 0x7ff68c102380 2024-04-26T21:59:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.585+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff684005020 con 0x7ff68c102380 2024-04-26T21:59:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.586+0000 7ff682a00640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 0x7ff6700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.586+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff6840d05b0 con 0x7ff68c102380 2024-04-26T21:59:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.586+0000 7ff689e00640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 0x7ff6700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.587+0000 7ff689e00640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 0x7ff6700a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff67c031500 tx=0x7ff67c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.590+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff68409d390 con 0x7ff68c102380 2024-04-26T21:59:39.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.809+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff68c063530 con 0x7ff68c102380 2024-04-26T21:59:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.810+0000 7ff682a00640 1 -- 172.21.15.12:0/2674424863 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18409 (secure 0 0 0) 0x7ff68409cc90 con 0x7ff68c102380 2024-04-26T21:59:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:39.811 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:58:46.347916+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 msgr2=0x7ff6700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 0x7ff6700a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff67c031500 tx=0x7ff67c005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 msgr2=0x7ff68c1a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff68400eb10 tx=0x7ff68400c2d0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 shutdown_connections 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff6700a3bb0 0x7ff6700a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c102380 0x7ff68c1a2140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff68c1019e0 0x7ff68c19b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 --2- 172.21.15.12:0/2674424863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68c101080 0x7ff68c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 >> 172.21.15.12:0/2674424863 conn(0x7ff68c0fcbe0 msgr2=0x7ff68c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 shutdown_connections 2024-04-26T21:59:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:39.813+0000 7ff68b200640 1 -- 172.21.15.12:0/2674424863 wait complete. 2024-04-26T21:59:39.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:59:39.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:59:39.893 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:39 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3714141261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 -- 172.21.15.12:0/1775175943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c104b90 msgr2=0x7fd00c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 --2- 172.21.15.12:0/1775175943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c104b90 0x7fd00c104fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009900 tx=0x7fcffc02a410 comp rx=0 tx=0).stop 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 -- 172.21.15.12:0/1775175943 shutdown_connections 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 --2- 172.21.15.12:0/1775175943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c106740 0x7fd00c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 --2- 172.21.15.12:0/1775175943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 0x7fd00c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 --2- 172.21.15.12:0/1775175943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c104b90 0x7fd00c104fb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 -- 172.21.15.12:0/1775175943 >> 172.21.15.12:0/1775175943 conn(0x7fd00c1000e0 msgr2=0x7fd00c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:40.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 -- 172.21.15.12:0/1775175943 shutdown_connections 2024-04-26T21:59:40.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.039+0000 7fd011c00640 1 -- 172.21.15.12:0/1775175943 wait complete. 2024-04-26T21:59:40.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 Processor -- start 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 -- start start 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 0x7fd00c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 0x7fd00c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 0x7fd00c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c110700 con 0x7fd00c104b90 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c110580 con 0x7fd00c106740 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.040+0000 7fd011c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c110880 con 0x7fd00c105d80 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 0x7fd00c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 0x7fd00c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34372/0 (socket says 172.21.15.12:34372) 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 0x7fd00c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00aa00640 1 -- 172.21.15.12:0/1159394508 learned_addr learned my addr 172.21.15.12:0/1159394508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:40.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd003e00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 0x7fd00c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 -- 172.21.15.12:0/1159394508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 msgr2=0x7fd00c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 0x7fd00c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 -- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 msgr2=0x7fd00c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 0x7fd00c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 -- 172.21.15.12:0/1159394508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c1a2a20 con 0x7fd00c106740 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.041+0000 7fd00b400640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 0x7fd00c1a23b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd00400ec50 tx=0x7fd00400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.042+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004003b70 con 0x7fd00c106740 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.042+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c1a2d10 con 0x7fd00c106740 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.042+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd004003d10 con 0x7fd00c106740 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.042+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c1a3200 con 0x7fd00c106740 2024-04-26T21:59:40.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.042+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004010830 con 0x7fd00c106740 2024-04-26T21:59:40.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.043+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c10bf50 con 0x7fd00c106740 2024-04-26T21:59:40.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.045+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd004003ec0 con 0x7fd00c106740 2024-04-26T21:59:40.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.046+0000 7fd003400640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 0x7fcff00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.047+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0040d04b0 con 0x7fd00c106740 2024-04-26T21:59:40.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.047+0000 7fd00aa00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 0x7fcff00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.048+0000 7fd00aa00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 0x7fcff00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcffc000c00 tx=0x7fcffc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.054+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd00409d290 con 0x7fd00c106740 2024-04-26T21:59:40.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:39 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3714141261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:40.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:39 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3714141261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:40.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.271+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd00c063530 con 0x7fd00c106740 2024-04-26T21:59:40.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.272+0000 7fd003400640 1 -- 172.21.15.12:0/1159394508 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd00409cb90 con 0x7fd00c106740 2024-04-26T21:59:40.271 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T21:59:40.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 msgr2=0x7fcff00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 0x7fcff00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcffc000c00 tx=0x7fcffc005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 msgr2=0x7fd00c1a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 0x7fd00c1a23b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd00400ec50 tx=0x7fd00400c6a0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 shutdown_connections 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcff00a3c30 0x7fcff00a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c106740 0x7fd00c1a23b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c105d80 0x7fd00c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 --2- 172.21.15.12:0/1159394508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd00c104b90 0x7fd00c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 >> 172.21.15.12:0/1159394508 conn(0x7fd00c1000e0 msgr2=0x7fd00c109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:40.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 shutdown_connections 2024-04-26T21:59:40.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.274+0000 7fd011c00640 1 -- 172.21.15.12:0/1159394508 wait complete. 2024-04-26T21:59:40.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 -- 172.21.15.12:0/1361481160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078530 msgr2=0x7f09f4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 --2- 172.21.15.12:0/1361481160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078530 0x7f09f4078990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009860 tx=0x7f09e002a340 comp rx=0 tx=0).stop 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 -- 172.21.15.12:0/1361481160 shutdown_connections 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 --2- 172.21.15.12:0/1361481160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f4078ed0 0x7f09f4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 --2- 172.21.15.12:0/1361481160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078530 0x7f09f4078990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 --2- 172.21.15.12:0/1361481160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f40772f0 0x7f09f4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 -- 172.21.15.12:0/1361481160 >> 172.21.15.12:0/1361481160 conn(0x7f09f4100120 msgr2=0x7f09f4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 -- 172.21.15.12:0/1361481160 shutdown_connections 2024-04-26T21:59:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.505+0000 7f09fbe00640 1 -- 172.21.15.12:0/1361481160 wait complete. 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.506+0000 7f09fbe00640 1 Processor -- start 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.506+0000 7f09fbe00640 1 -- start start 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f40772f0 0x7f09f419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078ed0 0x7f09f41a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4114af0 con 0x7f09f40772f0 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4114970 con 0x7f09f4078530 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09fbe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4114c70 con 0x7f09f4078ed0 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f40772f0 0x7f09f419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:33938/0 (socket says 172.21.15.12:33938) 2024-04-26T21:59:40.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 -- 172.21.15.12:0/65789570 learned_addr learned my addr 172.21.15.12:0/65789570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f9e00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078ed0 0x7f09f41a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 -- 172.21.15.12:0/65789570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078ed0 msgr2=0x7f09f41a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078ed0 0x7f09f41a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 -- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f40772f0 msgr2=0x7f09f419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f40772f0 0x7f09f419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.507+0000 7f09f8a00640 1 -- 172.21.15.12:0/65789570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f41a6e00 con 0x7f09f4078530 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09f8a00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09e00057c0 tx=0x7f09e0032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0037040 con 0x7f09f4078530 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09e0030bd0 con 0x7f09f4078530 2024-04-26T21:59:40.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41a7090 con 0x7f09f4078530 2024-04-26T21:59:40.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0005280 con 0x7f09f4078530 2024-04-26T21:59:40.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.508+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41a7520 con 0x7f09f4078530 2024-04-26T21:59:40.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.509+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f4111760 con 0x7f09f4078530 2024-04-26T21:59:40.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.511+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09e0043020 con 0x7f09f4078530 2024-04-26T21:59:40.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.513+0000 7f09f2000640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 0x7f09dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.513+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09e00ef5a0 con 0x7f09f4078530 2024-04-26T21:59:40.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.513+0000 7f09f9400640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 0x7f09dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.514+0000 7f09f9400640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 0x7f09dc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005fd0 tx=0x7f09e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.515+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f09e00bc380 con 0x7f09f4078530 2024-04-26T21:59:40.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.742+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f09f402dd20 con 0x7f09f4078530 2024-04-26T21:59:40.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.742+0000 7f09f2000640 1 -- 172.21.15.12:0/65789570 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f09e00328c0 con 0x7f09f4078530 2024-04-26T21:59:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:40.742 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-04-26T21:59:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T21:59:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-04-26T21:59:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-04-26T21:59:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T21:59:40.743 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 msgr2=0x7f09dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 0x7f09dc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09e8005fd0 tx=0x7f09e8009260 comp rx=0 tx=0).stop 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 msgr2=0x7f09f419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09e00057c0 tx=0x7f09e0032da0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 shutdown_connections 2024-04-26T21:59:40.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f09dc0a3bb0 0x7f09dc0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09f4078ed0 0x7f09f41a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f4078530 0x7f09f419f720 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 --2- 172.21.15.12:0/65789570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09f40772f0 0x7f09f419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 >> 172.21.15.12:0/65789570 conn(0x7f09f4100120 msgr2=0x7f09f41018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 shutdown_connections 2024-04-26T21:59:40.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.745+0000 7f09fbe00640 1 -- 172.21.15.12:0/65789570 wait complete. 2024-04-26T21:59:40.764 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:59:40.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:59:40.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 -- 172.21.15.12:0/3166823278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078ed0 msgr2=0x7f1610079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 --2- 172.21.15.12:0/3166823278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078ed0 0x7f1610079770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1608009ca0 tx=0x7f160802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 -- 172.21.15.12:0/3166823278 shutdown_connections 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 --2- 172.21.15.12:0/3166823278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078ed0 0x7f1610079770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 --2- 172.21.15.12:0/3166823278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1610078530 0x7f1610078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 --2- 172.21.15.12:0/3166823278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16100772f0 0x7f1610077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 -- 172.21.15.12:0/3166823278 >> 172.21.15.12:0/3166823278 conn(0x7f16100fffe0 msgr2=0x7f1610102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 -- 172.21.15.12:0/3166823278 shutdown_connections 2024-04-26T21:59:40.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.956+0000 7f1616400640 1 -- 172.21.15.12:0/3166823278 wait complete. 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.957+0000 7f1616400640 1 Processor -- start 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 -- start start 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078530 0x7f161019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 0x7f16101a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16101149f0 con 0x7f16100772f0 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1610114870 con 0x7f1610078530 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1616400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1610114b70 con 0x7f1610078ed0 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1615000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1615000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34414/0 (socket says 172.21.15.12:34414) 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1615000640 1 -- 172.21.15.12:0/3810257368 learned_addr learned my addr 172.21.15.12:0/3810257368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.958+0000 7f1607e00640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078530 0x7f161019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615a00640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 0x7f16101a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 -- 172.21.15.12:0/3810257368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 msgr2=0x7f16101a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 0x7f16101a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 -- 172.21.15.12:0/3810257368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078530 msgr2=0x7f161019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078530 0x7f161019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 -- 172.21.15.12:0/3810257368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16101a6c90 con 0x7f16100772f0 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615a00640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 0x7f16101a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f1615000640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f160000e9f0 tx=0x7f160000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.959+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160000cd80 con 0x7f16100772f0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.960+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16000108e0 con 0x7f16100772f0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.960+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16101a6f80 con 0x7f16100772f0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.960+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600018980 con 0x7f16100772f0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.960+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a73e0 con 0x7f16100772f0 2024-04-26T21:59:40.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.961+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16101113e0 con 0x7f16100772f0 2024-04-26T21:59:40.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.967+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1600010430 con 0x7f16100772f0 2024-04-26T21:59:40.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.968+0000 7f160e000640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 0x7f15f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:40.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.968+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f16000d0830 con 0x7f16100772f0 2024-04-26T21:59:40.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.968+0000 7f1607e00640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 0x7f15f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:40.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.969+0000 7f1607e00640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 0x7f15f40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16101a0580 tx=0x7f15f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:40.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:40.970+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f160009d610 con 0x7f16100772f0 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2674424863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1159394508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:41.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:40 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/65789570' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2674424863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1159394508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:41.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:40 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/65789570' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:59:41.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.185+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1610069ac0 con 0x7f16100772f0 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2674424863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:41.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1159394508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:41.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:40 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/65789570' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.991+0000 7f160e000640 1 -- 172.21.15.12:0/3810257368 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f160009cf10 con 0x7f16100772f0 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.995+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 msgr2=0x7f15f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.995+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 0x7f15f40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16101a0580 tx=0x7f15f8005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.995+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 msgr2=0x7f161019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.995+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f160000e9f0 tx=0x7f160000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 shutdown_connections 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15f40a3bb0 0x7f15f40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1610078ed0 0x7f16101a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1610078530 0x7f161019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 --2- 172.21.15.12:0/3810257368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16100772f0 0x7f161019f070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.996+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 >> 172.21.15.12:0/3810257368 conn(0x7f16100fffe0 msgr2=0x7f1610101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.997+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 shutdown_connections 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:41.998+0000 7f1616400640 1 -- 172.21.15.12:0/3810257368 wait complete. 2024-04-26T21:59:42.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T21:59:42.218 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:59:42.219 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:41 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:42.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 -- 172.21.15.12:0/2536847942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0101a70 msgr2=0x7f2ef0101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/2536847942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0101a70 0x7f2ef0101ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 -- 172.21.15.12:0/2536847942 shutdown_connections 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/2536847942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0102410 0x7f2ef0102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/2536847942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0101a70 0x7f2ef0101ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.235+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/2536847942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101110 0x7f2ef0101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.236+0000 7f2ef5a00640 1 -- 172.21.15.12:0/2536847942 >> 172.21.15.12:0/2536847942 conn(0x7f2ef00fccd0 msgr2=0x7f2ef00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.236+0000 7f2ef5a00640 1 -- 172.21.15.12:0/2536847942 shutdown_connections 2024-04-26T21:59:42.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.236+0000 7f2ef5a00640 1 -- 172.21.15.12:0/2536847942 wait complete. 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:41 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:42.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 Processor -- start 2024-04-26T21:59:42.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 -- start start 2024-04-26T21:59:42.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 0x7f2ef019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0102410 0x7f2ef01a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01105b0 con 0x7f2ef0101a70 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0110430 con 0x7f2ef0101110 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.237+0000 7f2ef5a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0110730 con 0x7f2ef0102410 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2ee7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 0x7f2ef019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:33976/0 (socket says 172.21.15.12:33976) 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2ee7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 0x7f2ef019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34442/0 (socket says 172.21.15.12:34442) 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 -- 172.21.15.12:0/1001617557 learned_addr learned my addr 172.21.15.12:0/1001617557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2ef5000640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0102410 0x7f2ef01a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 -- 172.21.15.12:0/1001617557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0102410 msgr2=0x7f2ef01a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0102410 0x7f2ef01a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 -- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 msgr2=0x7f2ef019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 0x7f2ef019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 -- 172.21.15.12:0/1001617557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01a2850 con 0x7f2ef0101110 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eefe00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9f0 tx=0x7f2ee000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.238+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000cd80 con 0x7f2ef0101110 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.239+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ee00108e0 con 0x7f2ef0101110 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.239+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01a2b40 con 0x7f2ef0101110 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.239+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0018980 con 0x7f2ef0101110 2024-04-26T21:59:42.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.239+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01a30e0 con 0x7f2ef0101110 2024-04-26T21:59:42.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.240+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef010c000 con 0x7f2ef0101110 2024-04-26T21:59:42.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.242+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ee0010430 con 0x7f2ef0101110 2024-04-26T21:59:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.243+0000 7f2eed600640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 0x7f2ed40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.243+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f2ee00d0460 con 0x7f2ef0101110 2024-04-26T21:59:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.243+0000 7f2ee7e00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 0x7f2ed40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:42.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.244+0000 7f2ee7e00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 0x7f2ed40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8031500 tx=0x7f2ed8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:42.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.247+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ee009ca80 con 0x7f2ef0101110 2024-04-26T21:59:42.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:42.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:42.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:42.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:42.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:59:42.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:41 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:42.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:42.475+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2ef002dad0 con 0x7f2ef0101110 2024-04-26T21:59:43.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.020+0000 7f2eed600640 1 -- 172.21.15.12:0/1001617557 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f2ef002dad0 con 0x7f2ef0101110 2024-04-26T21:59:43.021 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-04-26T21:59:43.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.027+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 msgr2=0x7f2ed40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.027+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 0x7f2ed40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8031500 tx=0x7f2ed8005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:43.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.028+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 msgr2=0x7f2ef019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.028+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9f0 tx=0x7f2ee000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 shutdown_connections 2024-04-26T21:59:43.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2ed40a3c30 0x7f2ed40a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef0102410 0x7f2ef01a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ef0101a70 0x7f2ef019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 --2- 172.21.15.12:0/1001617557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef0101110 0x7f2ef019ac40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.030+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 >> 172.21.15.12:0/1001617557 conn(0x7f2ef00fccd0 msgr2=0x7f2ef00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.031+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 shutdown_connections 2024-04-26T21:59:43.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.031+0000 7f2ef5a00640 1 -- 172.21.15.12:0/1001617557 wait complete. 2024-04-26T21:59:43.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:59:43.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 -- 172.21.15.12:0/4254731402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 msgr2=0x7faa04102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 --2- 172.21.15.12:0/4254731402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa04102d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009900 tx=0x7fa9f802a410 comp rx=0 tx=0).stop 2024-04-26T21:59:43.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 -- 172.21.15.12:0/4254731402 shutdown_connections 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 --2- 172.21.15.12:0/4254731402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04104520 0x7faa0410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 --2- 172.21.15.12:0/4254731402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04103b60 0x7faa04103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 --2- 172.21.15.12:0/4254731402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa04102d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 -- 172.21.15.12:0/4254731402 >> 172.21.15.12:0/4254731402 conn(0x7faa040fdee0 msgr2=0x7faa04100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 -- 172.21.15.12:0/4254731402 shutdown_connections 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.284+0000 7faa0b600640 1 -- 172.21.15.12:0/4254731402 wait complete. 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.285+0000 7faa0b600640 1 Processor -- start 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.285+0000 7faa0b600640 1 -- start start 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 0x7faa0419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 0x7faa041a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410e4f0 con 0x7faa04102970 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410e370 con 0x7faa04104520 2024-04-26T21:59:43.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa0b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410e670 con 0x7faa04103b60 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa08c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa03e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 0x7faa0419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa08c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34464/0 (socket says 172.21.15.12:34464) 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa08c00640 1 -- 172.21.15.12:0/1753834582 learned_addr learned my addr 172.21.15.12:0/1753834582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa09600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 0x7faa041a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.286+0000 7faa09600640 1 -- 172.21.15.12:0/1753834582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 msgr2=0x7faa0419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa09600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 0x7faa0419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa09600640 1 -- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 msgr2=0x7faa0419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa09600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa09600640 1 -- 172.21.15.12:0/1753834582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa041a2a30 con 0x7faa04104520 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa08c00640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa03e00640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 0x7faa0419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:43.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa09600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 0x7faa041a23c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009fd0 tx=0x7fa9fc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00ce20 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9fc005020 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa041a2d20 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc010690 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.287+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041a3210 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.289+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa041a34a0 con 0x7faa04104520 2024-04-26T21:59:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.290+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9fc0054d0 con 0x7faa04104520 2024-04-26T21:59:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.292+0000 7faa01600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.292+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fa9fc0d0980 con 0x7faa04104520 2024-04-26T21:59:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.292+0000 7faa08c00640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.293+0000 7faa08c00640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000c00 tx=0x7fa9f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.295+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa9fc09d470 con 0x7faa04104520 2024-04-26T21:59:43.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:43 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:59:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:43 smithi012 ceph-mon[28184]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:59:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:43 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1001617557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:43 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:43 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:59:43.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:43 smithi154 ceph-mon[32578]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:59:43.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:43 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1001617557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:43 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:43 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3810257368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:59:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:43 smithi077 ceph-mon[32280]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:59:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:43 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1001617557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:43 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T21:59:43.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:43.590+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faa041a3670 con 0x7faa04104520 2024-04-26T21:59:44.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.071+0000 7faa01600640 1 -- 172.21.15.12:0/1753834582 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa9fc09cd70 con 0x7faa04104520 2024-04-26T21:59:44.071 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:59:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.075+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 msgr2=0x7fa9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.075+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000c00 tx=0x7fa9f8005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.075+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 msgr2=0x7faa041a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.075+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 0x7faa041a23c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009fd0 tx=0x7fa9fc00ef10 comp rx=0 tx=0).stop 2024-04-26T21:59:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 shutdown_connections 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa04104520 0x7faa041a23c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa04103b60 0x7faa0419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 --2- 172.21.15.12:0/1753834582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa04102970 0x7faa0419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.076+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 >> 172.21.15.12:0/1753834582 conn(0x7faa040fdee0 msgr2=0x7faa04109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.077+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 shutdown_connections 2024-04-26T21:59:44.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.077+0000 7faa0b600640 1 -- 172.21.15.12:0/1753834582 wait complete. 2024-04-26T21:59:44.103 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T21:59:44.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T21:59:44.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 -- 172.21.15.12:0/2123185863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c102960 msgr2=0x7f405c102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 --2- 172.21.15.12:0/2123185863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c102960 0x7f405c102d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4050009900 tx=0x7f405002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 -- 172.21.15.12:0/2123185863 shutdown_connections 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 --2- 172.21.15.12:0/2123185863 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c104510 0x7f405c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 --2- 172.21.15.12:0/2123185863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c103b50 0x7f405c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 --2- 172.21.15.12:0/2123185863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c102960 0x7f405c102d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 -- 172.21.15.12:0/2123185863 >> 172.21.15.12:0/2123185863 conn(0x7f405c0fdef0 msgr2=0x7f405c100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 -- 172.21.15.12:0/2123185863 shutdown_connections 2024-04-26T21:59:44.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.292+0000 7f4064400640 1 -- 172.21.15.12:0/2123185863 wait complete. 2024-04-26T21:59:44.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.293+0000 7f4064400640 1 Processor -- start 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 -- start start 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c103b50 0x7f405c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c104510 0x7f405c1a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10e4d0 con 0x7f405c103b50 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10e350 con 0x7f405c104510 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4064400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10e650 con 0x7f405c102960 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4061a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4061000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c103b50 0x7f405c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4061a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:54384/0 (socket says 172.21.15.12:54384) 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4061a00640 1 -- 172.21.15.12:0/2176929380 learned_addr learned my addr 172.21.15.12:0/2176929380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.294+0000 7f4062400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c104510 0x7f405c1a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 -- 172.21.15.12:0/2176929380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c104510 msgr2=0x7f405c1a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c104510 0x7f405c1a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 -- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c103b50 msgr2=0x7f405c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c103b50 0x7f405c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 -- 172.21.15.12:0/2176929380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c1a2a40 con 0x7f405c102960 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4061a00640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4050031500 tx=0x7f4050003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405003f070 con 0x7f405c102960 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c1a2cd0 con 0x7f405c102960 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4050031c50 con 0x7f405c102960 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050035030 con 0x7f405c102960 2024-04-26T21:59:44.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.295+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c1a31c0 con 0x7f405c102960 2024-04-26T21:59:44.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.297+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c1a3390 con 0x7f405c102960 2024-04-26T21:59:44.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.299+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4050042070 con 0x7f405c102960 2024-04-26T21:59:44.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.300+0000 7f405a000640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 0x7f40440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.300+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f40500f09d0 con 0x7f405c102960 2024-04-26T21:59:44.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.300+0000 7f4061000640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 0x7f40440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.301+0000 7f4061000640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 0x7f40440a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f405c19c330 tx=0x7f4048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.304+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f40500bce10 con 0x7f405c102960 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:59:44.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: pgmap v314: 97 pgs: 3 creating+activating, 23 creating+peering, 70 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:44.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1753834582' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:44 smithi012 ceph-mon[28184]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:59:44.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: pgmap v314: 97 pgs: 3 creating+activating, 23 creating+peering, 70 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1753834582' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:44 smithi154 ceph-mon[32578]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:59:44.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: pgmap v314: 97 pgs: 3 creating+activating, 23 creating+peering, 70 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1753834582' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:59:44.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:44 smithi077 ceph-mon[32280]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:59:44.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:44.534+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f405c1a3720 con 0x7f405c102960 2024-04-26T21:59:45.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.027+0000 7f405a000640 1 -- 172.21.15.12:0/2176929380 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f40500c1110 con 0x7f405c102960 2024-04-26T21:59:45.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.030+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 msgr2=0x7f40440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.030+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 0x7f40440a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f405c19c330 tx=0x7f4048005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.030+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 msgr2=0x7f405c19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.030+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4050031500 tx=0x7f4050003c10 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 shutdown_connections 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f40440a3c30 0x7f40440a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405c104510 0x7f405c1a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f405c103b50 0x7f405c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 --2- 172.21.15.12:0/2176929380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f405c102960 0x7f405c19ae20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 >> 172.21.15.12:0/2176929380 conn(0x7f405c0fdef0 msgr2=0x7f405c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 shutdown_connections 2024-04-26T21:59:45.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.031+0000 7f4064400640 1 -- 172.21.15.12:0/2176929380 wait complete. 2024-04-26T21:59:45.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.286+0000 7f924fa00640 1 -- 172.21.15.12:0/2093144954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f924810a840 msgr2=0x7f924810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.286+0000 7f924fa00640 1 --2- 172.21.15.12:0/2093144954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f924810a840 0x7f924810ccd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9240009900 tx=0x7f924002a410 comp rx=0 tx=0).stop 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 -- 172.21.15.12:0/2093144954 shutdown_connections 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 --2- 172.21.15.12:0/2093144954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f924810a840 0x7f924810ccd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 --2- 172.21.15.12:0/2093144954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9248075390 0x7f9248075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 --2- 172.21.15.12:0/2093144954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248076970 0x7f9248074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 -- 172.21.15.12:0/2093144954 >> 172.21.15.12:0/2093144954 conn(0x7f92480fdf10 msgr2=0x7f9248100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 -- 172.21.15.12:0/2093144954 shutdown_connections 2024-04-26T21:59:45.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.287+0000 7f924fa00640 1 -- 172.21.15.12:0/2093144954 wait complete. 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.288+0000 7f924fa00640 1 Processor -- start 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.288+0000 7f924fa00640 1 -- start start 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 0x7f924819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 0x7f92481a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9248110670 con 0x7f9248076970 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92481104f0 con 0x7f9248075390 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924fa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92481107f0 con 0x7f924810a840 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924d000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 0x7f924819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34486/0 (socket says 172.21.15.12:34486) 2024-04-26T21:59:45.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 -- 172.21.15.12:0/1530953783 learned_addr learned my addr 172.21.15.12:0/1530953783 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:45.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924d000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 0x7f924819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:34032/0 (socket says 172.21.15.12:34032) 2024-04-26T21:59:45.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f924da00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 0x7f92481a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 -- 172.21.15.12:0/1530953783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 msgr2=0x7f92481a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 0x7f92481a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 -- 172.21.15.12:0/1530953783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 msgr2=0x7f924819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 0x7f924819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.289+0000 7f9247e00640 1 -- 172.21.15.12:0/1530953783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92481a2980 con 0x7f9248076970 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f924da00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 0x7f92481a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f9247e00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f923000ea60 tx=0x7f923000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000cde0 con 0x7f9248076970 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92300035f0 con 0x7f9248076970 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230010690 con 0x7f9248076970 2024-04-26T21:59:45.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92481a2c70 con 0x7f9248076970 2024-04-26T21:59:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.290+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a31c0 con 0x7f9248076970 2024-04-26T21:59:45.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.292+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9248109570 con 0x7f9248076970 2024-04-26T21:59:45.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.294+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f92300037a0 con 0x7f9248076970 2024-04-26T21:59:45.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.295+0000 7f9245600640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 0x7f922c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.296+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f92300d06d0 con 0x7f9248076970 2024-04-26T21:59:45.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.296+0000 7f924d000640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 0x7f922c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.296+0000 7f924d000640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 0x7f922c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9238005fd0 tx=0x7f9238009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.300+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f923009d020 con 0x7f9248076970 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2176929380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:45 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2176929380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:45 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:59:45.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:45.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2176929380' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:45 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:45.509+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9248063560 con 0x7f9248076970 2024-04-26T21:59:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T21:59:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:59:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:active 2024-04-26T21:59:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: pgmap v317: 97 pgs: 3 creating+activating, 29 creating+peering, 36 unknown, 29 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:46.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:46 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:59:46.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:active 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: pgmap v317: 97 pgs: 3 creating+activating, 29 creating+peering, 36 unknown, 29 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:46.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:46 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:59:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T21:59:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:59:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] up:active 2024-04-26T21:59:46.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:46.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: pgmap v317: 97 pgs: 3 creating+activating, 29 creating+peering, 36 unknown, 29 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:46.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:46 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:59:47.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.099+0000 7f9245600640 1 -- 172.21.15.12:0/1530953783 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f923009c920 con 0x7f9248076970 2024-04-26T21:59:47.100 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 msgr2=0x7f922c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 0x7f922c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9238005fd0 tx=0x7f9238009500 comp rx=0 tx=0).stop 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 msgr2=0x7f924819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f923000ea60 tx=0x7f923000ef20 comp rx=0 tx=0).stop 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 shutdown_connections 2024-04-26T21:59:47.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f922c0a3c30 0x7f922c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f924810a840 0x7f92481a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076970 0x7f924819b2a0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f923000ea60 tx=0x7f923000ef20 comp rx=0 tx=0).stop 2024-04-26T21:59:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 --2- 172.21.15.12:0/1530953783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9248075390 0x7f924819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.104+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 >> 172.21.15.12:0/1530953783 conn(0x7f92480fdf10 msgr2=0x7f9248106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:47.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.106+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 shutdown_connections 2024-04-26T21:59:47.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.108+0000 7f924fa00640 1 -- 172.21.15.12:0/1530953783 wait complete. 2024-04-26T21:59:47.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T21:59:47.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:47 smithi012 ceph-mon[28184]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:59:47.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:47 smithi012 ceph-mon[28184]: Cluster is now healthy 2024-04-26T21:59:47.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:47 smithi012 ceph-mon[28184]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 -- 172.21.15.12:0/3930956607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101260 msgr2=0x7f1b04101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 --2- 172.21.15.12:0/3930956607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101260 0x7f1b04101680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af402a950 comp rx=0 tx=0).stop 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 -- 172.21.15.12:0/3930956607 shutdown_connections 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 --2- 172.21.15.12:0/3930956607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 --2- 172.21.15.12:0/3930956607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101bc0 0x7f1b04102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 --2- 172.21.15.12:0/3930956607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101260 0x7f1b04101680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 -- 172.21.15.12:0/3930956607 >> 172.21.15.12:0/3930956607 conn(0x7f1b040fcde0 msgr2=0x7f1b040ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 -- 172.21.15.12:0/3930956607 shutdown_connections 2024-04-26T21:59:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.327+0000 7f1b0a000640 1 -- 172.21.15.12:0/3930956607 wait complete. 2024-04-26T21:59:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.328+0000 7f1b0a000640 1 Processor -- start 2024-04-26T21:59:47.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 -- start start 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 0x7f1b0419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101bc0 0x7f1b0419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b041106a0 con 0x7f1b04102560 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b04110520 con 0x7f1b04101260 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b0a000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b04110820 con 0x7f1b04101bc0 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.329+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 0x7f1b0419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34504/0 (socket says 172.21.15.12:34504) 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 -- 172.21.15.12:0/1121173368 learned_addr learned my addr 172.21.15.12:0/1121173368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:47.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1afaa00640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101bc0 0x7f1b0419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 -- 172.21.15.12:0/1121173368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101bc0 msgr2=0x7f1b0419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101bc0 0x7f1b0419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 -- 172.21.15.12:0/1121173368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 msgr2=0x7f1b0419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 0x7f1b0419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 -- 172.21.15.12:0/1121173368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b041a2a20 con 0x7f1b04102560 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03400640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 0x7f1b0419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:47.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.330+0000 7f1b03e00640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e9b0 tx=0x7f1afc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.331+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00cd80 con 0x7f1b04102560 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.331+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b041a2d10 con 0x7f1b04102560 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.331+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1afc0035f0 con 0x7f1b04102560 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.331+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc010690 con 0x7f1b04102560 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.331+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b041a3200 con 0x7f1b04102560 2024-04-26T21:59:47.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.332+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0410c240 con 0x7f1b04102560 2024-04-26T21:59:47.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.334+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1afc004000 con 0x7f1b04102560 2024-04-26T21:59:47.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.336+0000 7f1b00c00640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 0x7f1ae80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.336+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f1afc0d0a00 con 0x7f1b04102560 2024-04-26T21:59:47.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.336+0000 7f1b03400640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 0x7f1ae80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.337+0000 7f1b03400640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 0x7f1ae80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1af4000f80 tx=0x7f1af402f890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:47.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.338+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1afc09d1f0 con 0x7f1b04102560 2024-04-26T21:59:47.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:47 smithi154 ceph-mon[32578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:59:47.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:47 smithi154 ceph-mon[32578]: Cluster is now healthy 2024-04-26T21:59:47.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:47 smithi154 ceph-mon[32578]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:59:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:47 smithi077 ceph-mon[32280]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T21:59:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:47 smithi077 ceph-mon[32280]: Cluster is now healthy 2024-04-26T21:59:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:47 smithi077 ceph-mon[32280]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:59:47.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:47.539+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f1b0402dad0 con 0x7f1b04102560 2024-04-26T21:59:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T21:59:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.111+0000 7f1b00c00640 1 -- 172.21.15.12:0/1121173368 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f1afc09caf0 con 0x7f1b04102560 2024-04-26T21:59:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.115+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 msgr2=0x7f1ae80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.115+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 0x7f1ae80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1af4000f80 tx=0x7f1af402f890 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.115+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 msgr2=0x7f1b041a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.115+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e9b0 tx=0x7f1afc00ee70 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 shutdown_connections 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1ae80a3bb0 0x7f1ae80a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b04102560 0x7f1b041a23b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b04101bc0 0x7f1b0419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 --2- 172.21.15.12:0/1121173368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b04101260 0x7f1b0419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.116+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 >> 172.21.15.12:0/1121173368 conn(0x7f1b040fcde0 msgr2=0x7f1b040fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.117+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 shutdown_connections 2024-04-26T21:59:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.118+0000 7f1b0a000640 1 -- 172.21.15.12:0/1121173368 wait complete. 2024-04-26T21:59:48.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T21:59:48.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:48 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:59:48.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:48 smithi077 ceph-mon[32280]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:59:48.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:48 smithi077 ceph-mon[32280]: pgmap v320: 129 pgs: 3 creating+activating, 29 creating+peering, 51 unknown, 46 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:48.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:48 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T21:59:48.233 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 21:59:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: 2024-04-26T21:59:47.885+0000 7fc84d480700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:59:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 -- 172.21.15.12:0/306281417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8103360 msgr2=0x7f91e8105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 --2- 172.21.15.12:0/306281417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8103360 0x7f91e8105750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009ca0 tx=0x7f91d802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 -- 172.21.15.12:0/306281417 shutdown_connections 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 --2- 172.21.15.12:0/306281417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e81085c0 0x7f91e810a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 --2- 172.21.15.12:0/306281417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e8105c90 0x7f91e8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 --2- 172.21.15.12:0/306281417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8103360 0x7f91e8105750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.328+0000 7f91ed400640 1 -- 172.21.15.12:0/306281417 >> 172.21.15.12:0/306281417 conn(0x7f91e80fccd0 msgr2=0x7f91e80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.329+0000 7f91ed400640 1 -- 172.21.15.12:0/306281417 shutdown_connections 2024-04-26T21:59:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.329+0000 7f91ed400640 1 -- 172.21.15.12:0/306281417 wait complete. 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 Processor -- start 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 -- start start 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 0x7f91e8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e81085c0 0x7f91e8101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810e3e0 con 0x7f91e8103360 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810e260 con 0x7f91e8105c90 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91ed400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810e560 con 0x7f91e81085c0 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.330+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34514/0 (socket says 172.21.15.12:34514) 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 -- 172.21.15.12:0/469403657 learned_addr learned my addr 172.21.15.12:0/469403657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91df400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 0x7f91e8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91eca00640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e81085c0 0x7f91e8101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 -- 172.21.15.12:0/469403657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e81085c0 msgr2=0x7f91e8101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e81085c0 0x7f91e8101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 -- 172.21.15.12:0/469403657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 msgr2=0x7f91e8101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 0x7f91e8101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91e7e00640 1 -- 172.21.15.12:0/469403657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e81020e0 con 0x7f91e8103360 2024-04-26T21:59:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.331+0000 7f91df400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 0x7f91e8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91e7e00640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91d8005e30 tx=0x7f91d80317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8003bf0 con 0x7f91e8103360 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d8003d90 con 0x7f91e8103360 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d802fc10 con 0x7f91e8103360 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e81a2bb0 con 0x7f91e8103360 2024-04-26T21:59:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.332+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81a30b0 con 0x7f91e8103360 2024-04-26T21:59:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.333+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e80fe7c0 con 0x7f91e8103360 2024-04-26T21:59:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.337+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f91d8004040 con 0x7f91e8103360 2024-04-26T21:59:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.339+0000 7f91e5600640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 0x7f91cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.339+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f91d80f1a50 con 0x7f91e8103360 2024-04-26T21:59:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.339+0000 7f91df400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 0x7f91cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.340+0000 7f91df400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 0x7f91cc0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91e8102a40 tx=0x7f91d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.343+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f91d80be240 con 0x7f91e8103360 2024-04-26T21:59:48.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:48 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:59:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:48 smithi154 ceph-mon[32578]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:59:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:48 smithi154 ceph-mon[32578]: pgmap v320: 129 pgs: 3 creating+activating, 29 creating+peering, 51 unknown, 46 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:48 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T21:59:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:48 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1530953783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:59:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:48 smithi012 ceph-mon[28184]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:59:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:48 smithi012 ceph-mon[28184]: pgmap v320: 129 pgs: 3 creating+activating, 29 creating+peering, 51 unknown, 46 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-26T21:59:48.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:48 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T21:59:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:48.586+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f91e802dad0 con 0x7f91e8103360 2024-04-26T21:59:49.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.129+0000 7f91e5600640 1 -- 172.21.15.12:0/469403657 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f91d80bdb40 con 0x7f91e8103360 2024-04-26T21:59:49.129 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 4 to fsmap 2024-04-26T21:59:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.133+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 msgr2=0x7f91cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.133+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 0x7f91cc0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91e8102a40 tx=0x7f91d0005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.133+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 msgr2=0x7f91e8102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.133+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91d8005e30 tx=0x7f91d80317e0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.133+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 shutdown_connections 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f91cc0a3bb0 0x7f91cc0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91e81085c0 0x7f91e8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91e8105c90 0x7f91e8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 --2- 172.21.15.12:0/469403657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91e8103360 0x7f91e8102cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 >> 172.21.15.12:0/469403657 conn(0x7f91e80fccd0 msgr2=0x7f91e8103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 shutdown_connections 2024-04-26T21:59:49.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.134+0000 7f91ed400640 1 -- 172.21.15.12:0/469403657 wait complete. 2024-04-26T21:59:49.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.366+0000 7f8e26200640 1 -- 172.21.15.12:0/3574882221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20109f80 msgr2=0x7f8e2010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.366+0000 7f8e26200640 1 --2- 172.21.15.12:0/3574882221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20109f80 0x7f8e2010a400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009900 tx=0x7f8e0c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.366+0000 7f8e26200640 1 -- 172.21.15.12:0/3574882221 shutdown_connections 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 --2- 172.21.15.12:0/3574882221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 --2- 172.21.15.12:0/3574882221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20109f80 0x7f8e2010a400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 --2- 172.21.15.12:0/3574882221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20108d90 0x7f8e201091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 -- 172.21.15.12:0/3574882221 >> 172.21.15.12:0/3574882221 conn(0x7f8e20077d30 msgr2=0x7f8e2007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 -- 172.21.15.12:0/3574882221 shutdown_connections 2024-04-26T21:59:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 -- 172.21.15.12:0/3574882221 wait complete. 2024-04-26T21:59:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.367+0000 7f8e26200640 1 Processor -- start 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 -- start start 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20108d90 0x7f8e2019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201148a0 con 0x7f8e20109f80 2024-04-26T21:59:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114720 con 0x7f8e2010a940 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e26200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114a20 con 0x7f8e20108d90 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e1fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e1fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34048/0 (socket says 172.21.15.12:34048) 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e1fe00640 1 -- 172.21.15.12:0/1781394527 learned_addr learned my addr 172.21.15.12:0/1781394527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e25800640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.368+0000 7f8e24e00640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20108d90 0x7f8e2019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 -- 172.21.15.12:0/1781394527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20108d90 msgr2=0x7f8e2019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20108d90 0x7f8e2019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 -- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 msgr2=0x7f8e2019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 -- 172.21.15.12:0/1781394527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201a6bc0 con 0x7f8e2010a940 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:49.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e25800640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201a6550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800da10 tx=0x7f8e1800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800bd70 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e18003e20 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180106c0 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.369+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e201a6eb0 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.370+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e201a7320 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.371+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e20110120 con 0x7f8e2010a940 2024-04-26T21:59:49.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.373+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e18004270 con 0x7f8e2010a940 2024-04-26T21:59:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.374+0000 7f8e1d600640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 0x7f8e080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.374+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8e180d0a30 con 0x7f8e2010a940 2024-04-26T21:59:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.374+0000 7f8e24e00640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 0x7f8e080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.375+0000 7f8e24e00640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 0x7f8e080a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8e20109de0 tx=0x7f8e14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.378+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e1809d170 con 0x7f8e2010a940 2024-04-26T21:59:49.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:49 smithi154 ceph-mon[32578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:49.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:49 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T21:59:49.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:49 smithi154 ceph-mon[32578]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:59:49.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:49 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T21:59:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:49 smithi077 ceph-mon[32280]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:49 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T21:59:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:49 smithi077 ceph-mon[32280]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:59:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:49 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T21:59:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:49 smithi012 ceph-mon[28184]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:59:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:49 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1121173368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T21:59:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:49 smithi012 ceph-mon[28184]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:59:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:49 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T21:59:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.630+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e2002dad0 con 0x7f8e2010a940 2024-04-26T21:59:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.632+0000 7f8e1d600640 1 -- 172.21.15.12:0/1781394527 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7f8e1809ca70 con 0x7f8e2010a940 2024-04-26T21:59:49.632 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:49.632 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T21:59:48.588596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T21:59:49.633 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.634+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 msgr2=0x7f8e080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.634+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 0x7f8e080a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8e20109de0 tx=0x7f8e14005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.634+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 msgr2=0x7f8e201a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.634+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201a6550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800da10 tx=0x7f8e1800ded0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 shutdown_connections 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8e080a3bb0 0x7f8e080a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e2010a940 0x7f8e201a6550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109f80 0x7f8e2019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 --2- 172.21.15.12:0/1781394527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e20108d90 0x7f8e2019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 >> 172.21.15.12:0/1781394527 conn(0x7f8e20077d30 msgr2=0x7f8e2010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:49.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 shutdown_connections 2024-04-26T21:59:49.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.635+0000 7f8e26200640 1 -- 172.21.15.12:0/1781394527 wait complete. 2024-04-26T21:59:49.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 -- 172.21.15.12:0/2514127771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838104960 msgr2=0x7f3838104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 --2- 172.21.15.12:0/2514127771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838104960 0x7f3838104d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3828009900 tx=0x7f382802a410 comp rx=0 tx=0).stop 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 -- 172.21.15.12:0/2514127771 shutdown_connections 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 --2- 172.21.15.12:0/2514127771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838106510 0x7f383810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 --2- 172.21.15.12:0/2514127771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f3838105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 --2- 172.21.15.12:0/2514127771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838104960 0x7f3838104d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 -- 172.21.15.12:0/2514127771 >> 172.21.15.12:0/2514127771 conn(0x7f38380ffeb0 msgr2=0x7f3838102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:49.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 -- 172.21.15.12:0/2514127771 shutdown_connections 2024-04-26T21:59:49.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.860+0000 7f3837200640 1 -- 172.21.15.12:0/2514127771 wait complete. 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 Processor -- start 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 -- start start 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838104960 0x7f383819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 0x7f38381a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38381104e0 con 0x7f3838106510 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3838110360 con 0x7f3838105b50 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.861+0000 7f3837200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3838110660 con 0x7f3838104960 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f382fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 0x7f38381a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f382fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:37926/0 (socket says 172.21.15.12:37926) 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f382fe00640 1 -- 172.21.15.12:0/765042265 learned_addr learned my addr 172.21.15.12:0/765042265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:49.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3835e00640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838104960 0x7f383819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 -- 172.21.15.12:0/765042265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838104960 msgr2=0x7f383819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838104960 0x7f383819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 -- 172.21.15.12:0/765042265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 msgr2=0x7f383819b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f3836800640 1 -- 172.21.15.12:0/765042265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a2770 con 0x7f3838106510 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.862+0000 7f382fe00640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f3836800640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 0x7f38381a2100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3830009fd0 tx=0x7f383000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000ce20 con 0x7f3838106510 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830005020 con 0x7f3838106510 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a2a60 con 0x7f3838106510 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830010690 con 0x7f3838106510 2024-04-26T21:59:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.863+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a3000 con 0x7f3838106510 2024-04-26T21:59:49.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.864+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f383810bc80 con 0x7f3838106510 2024-04-26T21:59:49.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.867+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38300054d0 con 0x7f3838106510 2024-04-26T21:59:49.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.868+0000 7f382ea00640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 0x7f381c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:49.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.868+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f38300d0c90 con 0x7f3838106510 2024-04-26T21:59:49.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.868+0000 7f3835e00640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 0x7f381c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:49.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.869+0000 7f3835e00640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 0x7f381c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3828005ab0 tx=0x7f3828005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:49.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:49.872+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f383009d3d0 con 0x7f3838106510 2024-04-26T21:59:50.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.079+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38380635d0 con 0x7f3838106510 2024-04-26T21:59:50.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.084+0000 7f382ea00640 1 -- 172.21.15.12:0/765042265 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f383009ccd0 con 0x7f3838106510 2024-04-26T21:59:50.085 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:50.086 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 msgr2=0x7f381c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 0x7f381c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3828005ab0 tx=0x7f3828005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 msgr2=0x7f38381a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 0x7f38381a2100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3830009fd0 tx=0x7f383000ef10 comp rx=0 tx=0).stop 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 shutdown_connections 2024-04-26T21:59:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f381c0a3c30 0x7f381c0a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3838106510 0x7f38381a2100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3838105b50 0x7f383819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 --2- 172.21.15.12:0/765042265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3838104960 0x7f383819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 >> 172.21.15.12:0/765042265 conn(0x7f38380ffeb0 msgr2=0x7f3838108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 shutdown_connections 2024-04-26T21:59:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.087+0000 7f3837200640 1 -- 172.21.15.12:0/765042265 wait complete. 2024-04-26T21:59:50.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: pgmap v323: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1781394527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/765042265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.322+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3665196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 msgr2=0x7f0f54102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.322+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3665196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f54102bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009860 tx=0x7f0f4402a340 comp rx=0 tx=0).stop 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3665196536 shutdown_connections 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3665196536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 0x7f0f5410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3665196536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 0x7f0f54103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3665196536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f54102bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3665196536 >> 172.21.15.12:0/3665196536 conn(0x7f0f540fdd40 msgr2=0x7f0f54100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3665196536 shutdown_connections 2024-04-26T21:59:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.323+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3665196536 wait complete. 2024-04-26T21:59:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.324+0000 7f0f5ae00640 1 Processor -- start 2024-04-26T21:59:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.324+0000 7f0f5ae00640 1 -- start start 2024-04-26T21:59:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.324+0000 7f0f5ae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.324+0000 7f0f5ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 0x7f0f5419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.324+0000 7f0f5ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 0x7f0f541a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f5ae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410e2b0 con 0x7f0f541027b0 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f5ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410e130 con 0x7f0f541039a0 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f5ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410e430 con 0x7f0f54104360 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34068/0 (socket says 172.21.15.12:34068) 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 -- 172.21.15.12:0/3332275960 learned_addr learned my addr 172.21.15.12:0/3332275960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59000640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 0x7f0f5419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f5a400640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 0x7f0f541a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 -- 172.21.15.12:0/3332275960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 msgr2=0x7f0f541a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 0x7f0f541a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 -- 172.21.15.12:0/3332275960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 msgr2=0x7f0f5419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 0x7f0f5419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.325+0000 7f0f59a00640 1 -- 172.21.15.12:0/3332275960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541a27a0 con 0x7f0f541027b0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f59a00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f440057c0 tx=0x7f0f44032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44037040 con 0x7f0f541027b0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541a2a30 con 0x7f0f541027b0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44030bd0 con 0x7f0f541027b0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44005280 con 0x7f0f541027b0 2024-04-26T21:59:50.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.326+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a2f10 con 0x7f0f541027b0 2024-04-26T21:59:50.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.327+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f541a3140 con 0x7f0f541027b0 2024-04-26T21:59:50.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.330+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f44043020 con 0x7f0f541027b0 2024-04-26T21:59:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.331+0000 7f0f52000640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 0x7f0f380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.331+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0f440efb60 con 0x7f0f541027b0 2024-04-26T21:59:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.331+0000 7f0f59000640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 0x7f0f380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.332+0000 7f0f59000640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 0x7f0f380a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f5419c090 tx=0x7f0f40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.336+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f440bc2a0 con 0x7f0f541027b0 2024-04-26T21:59:50.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: pgmap v323: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-26T21:59:50.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:59:50.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T21:59:50.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:50.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1781394527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/765042265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.547+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f541a34d0 con 0x7f0f541027b0 2024-04-26T21:59:50.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.550+0000 7f0f52000640 1 -- 172.21.15.12:0/3332275960 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f0f440bbba0 con 0x7f0f541027b0 2024-04-26T21:59:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:50.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 msgr2=0x7f0f380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 0x7f0f380a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0f5419c090 tx=0x7f0f40005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:50.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 msgr2=0x7f0f5419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f440057c0 tx=0x7f0f44032da0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 shutdown_connections 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f0f380a3c30 0x7f0f380a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f54104360 0x7f0f541a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f541039a0 0x7f0f5419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 --2- 172.21.15.12:0/3332275960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f541027b0 0x7f0f5419ab80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 >> 172.21.15.12:0/3332275960 conn(0x7f0f540fdd40 msgr2=0x7f0f54109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 shutdown_connections 2024-04-26T21:59:50.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.553+0000 7f0f5ae00640 1 -- 172.21.15.12:0/3332275960 wait complete. 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: pgmap v323: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/469403657' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1781394527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/765042265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:50.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:50.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 -- 172.21.15.12:0/4215611434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08103220 msgr2=0x7f4c08109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 --2- 172.21.15.12:0/4215611434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08103220 0x7f4c08109ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 -- 172.21.15.12:0/4215611434 shutdown_connections 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 --2- 172.21.15.12:0/4215611434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08103220 0x7f4c08109ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 --2- 172.21.15.12:0/4215611434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 0x7f4c08102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 --2- 172.21.15.12:0/4215611434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08101670 0x7f4c08101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 -- 172.21.15.12:0/4215611434 >> 172.21.15.12:0/4215611434 conn(0x7f4c080fcbc0 msgr2=0x7f4c080ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.795+0000 7f4c07200640 1 -- 172.21.15.12:0/4215611434 shutdown_connections 2024-04-26T21:59:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.796+0000 7f4c07200640 1 -- 172.21.15.12:0/4215611434 wait complete. 2024-04-26T21:59:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 Processor -- start 2024-04-26T21:59:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 -- start start 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 0x7f4c08198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08103220 0x7f4c0819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e2c0 con 0x7f4c08101670 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e140 con 0x7f4c08103220 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c07200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e440 con 0x7f4c08102860 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c05e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c05e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34084/0 (socket says 172.21.15.12:34084) 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.797+0000 7f4c05e00640 1 -- 172.21.15.12:0/102362294 learned_addr learned my addr 172.21.15.12:0/102362294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c06800640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08103220 0x7f4c0819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05400640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 0x7f4c08198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 -- 172.21.15.12:0/102362294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 msgr2=0x7f4c08198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 0x7f4c08198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 -- 172.21.15.12:0/102362294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08103220 msgr2=0x7f4c0819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08103220 0x7f4c0819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 -- 172.21.15.12:0/102362294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c081a0500 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4c05e00640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800ebc0 tx=0x7f4bf800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8003b70 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf8003d10 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.798+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8019800 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.799+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c081a07f0 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.799+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a0d40 con 0x7f4c08101670 2024-04-26T21:59:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.800+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08108930 con 0x7f4c08101670 2024-04-26T21:59:50.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.802+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4bf80199a0 con 0x7f4c08101670 2024-04-26T21:59:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.803+0000 7f4bfea00640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 0x7f4bec0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.803+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4bf80d0ea0 con 0x7f4c08101670 2024-04-26T21:59:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.804+0000 7f4c05400640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 0x7f4bec0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.804+0000 7f4c05400640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 0x7f4bec0a61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c08199df0 tx=0x7f4bf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:50.805+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4bf809d6f0 con 0x7f4c08101670 2024-04-26T21:59:51.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.050+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c0802dad0 con 0x7f4c08101670 2024-04-26T21:59:51.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.052+0000 7f4bfea00640 1 -- 172.21.15.12:0/102362294 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7f4bf809cff0 con 0x7f4c08101670 2024-04-26T21:59:51.052 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:51.052 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T21:59:48.588596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T21:59:51.053 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-04-26T21:59:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 msgr2=0x7f4bec0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 0x7f4bec0a61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c08199df0 tx=0x7f4bf0005f90 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 msgr2=0x7f4c081988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800ebc0 tx=0x7f4bf800c6a0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 shutdown_connections 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4bec0a3d00 0x7f4bec0a61c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08103220 0x7f4c0819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c08102860 0x7f4c08198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 --2- 172.21.15.12:0/102362294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c08101670 0x7f4c081988e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 >> 172.21.15.12:0/102362294 conn(0x7f4c080fcbc0 msgr2=0x7f4c08105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 shutdown_connections 2024-04-26T21:59:51.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.055+0000 7f4c07200640 1 -- 172.21.15.12:0/102362294 wait complete. 2024-04-26T21:59:51.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:51.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 -- 172.21.15.12:0/1323229615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8105ba0 msgr2=0x7f7de8106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 --2- 172.21.15.12:0/1323229615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8105ba0 0x7f7de8106020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009900 tx=0x7f7dd402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 -- 172.21.15.12:0/1323229615 shutdown_connections 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 --2- 172.21.15.12:0/1323229615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de8106560 0x7f7de810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 --2- 172.21.15.12:0/1323229615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8105ba0 0x7f7de8106020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 --2- 172.21.15.12:0/1323229615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de81049b0 0x7f7de8104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 -- 172.21.15.12:0/1323229615 >> 172.21.15.12:0/1323229615 conn(0x7f7de80fff40 msgr2=0x7f7de8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 -- 172.21.15.12:0/1323229615 shutdown_connections 2024-04-26T21:59:51.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.286+0000 7f7dee800640 1 -- 172.21.15.12:0/1323229615 wait complete. 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.287+0000 7f7dee800640 1 Processor -- start 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.287+0000 7f7dee800640 1 -- start start 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de81049b0 0x7f7de819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de8105ba0 0x7f7de819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8110510 con 0x7f7de8106560 2024-04-26T21:59:51.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8110390 con 0x7f7de81049b0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dee800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8110690 con 0x7f7de8105ba0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7ded400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de81049b0 0x7f7de819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34108/0 (socket says 172.21.15.12:34108) 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 -- 172.21.15.12:0/2199091019 learned_addr learned my addr 172.21.15.12:0/2199091019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7deca00640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de8105ba0 0x7f7de819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 -- 172.21.15.12:0/2199091019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de8105ba0 msgr2=0x7f7de819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de8105ba0 0x7f7de819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 -- 172.21.15.12:0/2199091019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de81049b0 msgr2=0x7f7de819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de81049b0 0x7f7de819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.288+0000 7f7dede00640 1 -- 172.21.15.12:0/2199091019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de81a27a0 con 0x7f7de8106560 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7dede00640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7de000d950 tx=0x7f7de000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0017030 con 0x7f7de8106560 2024-04-26T21:59:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de0004020 con 0x7f7de8106560 2024-04-26T21:59:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0003b50 con 0x7f7de8106560 2024-04-26T21:59:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81a2a90 con 0x7f7de8106560 2024-04-26T21:59:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.289+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81a2f60 con 0x7f7de8106560 2024-04-26T21:59:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.291+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de810bcd0 con 0x7f7de8106560 2024-04-26T21:59:51.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.292+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7de001a070 con 0x7f7de8106560 2024-04-26T21:59:51.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.294+0000 7f7de6000640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 0x7f7dcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.294+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7de00cfcc0 con 0x7f7de8106560 2024-04-26T21:59:51.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.294+0000 7f7ded400640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 0x7f7dcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.295+0000 7f7ded400640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 0x7f7dcc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7de8105a00 tx=0x7f7dd8009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.298+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7de0021070 con 0x7f7de8106560 2024-04-26T21:59:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:51 smithi012 ceph-mon[28184]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:59:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:51 smithi012 ceph-mon[28184]: Cluster is now healthy 2024-04-26T21:59:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:51 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3332275960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:51 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/102362294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:51 smithi154 ceph-mon[32578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:59:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:51 smithi154 ceph-mon[32578]: Cluster is now healthy 2024-04-26T21:59:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:51 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3332275960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:51 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/102362294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:51 smithi077 ceph-mon[32280]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:59:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:51 smithi077 ceph-mon[32280]: Cluster is now healthy 2024-04-26T21:59:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:51 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3332275960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:51 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/102362294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:51.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.506+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7de802db20 con 0x7f7de8106560 2024-04-26T21:59:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.510+0000 7f7de6000640 1 -- 172.21.15.12:0/2199091019 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f7de009c460 con 0x7f7de8106560 2024-04-26T21:59:51.511 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T21:59:51.511 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 msgr2=0x7f7dcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 0x7f7dcc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7de8105a00 tx=0x7f7dd8009670 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 msgr2=0x7f7de81a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7de000d950 tx=0x7f7de000de10 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 shutdown_connections 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dcc0a3c30 0x7f7dcc0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7de8106560 0x7f7de81a2130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7de8105ba0 0x7f7de819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 --2- 172.21.15.12:0/2199091019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7de81049b0 0x7f7de819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 >> 172.21.15.12:0/2199091019 conn(0x7f7de80fff40 msgr2=0x7f7de8108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 shutdown_connections 2024-04-26T21:59:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.513+0000 7f7dee800640 1 -- 172.21.15.12:0/2199091019 wait complete. 2024-04-26T21:59:51.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:59:43.592535+0000', 'modified': '2024-04-26T21:59:48.588596+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6839/4217479283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4217479283}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4217479283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-26T21:59:51.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:59:51.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:59:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 -- 172.21.15.12:0/27475493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900106530 msgr2=0x7fc90010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 --2- 172.21.15.12:0/27475493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900106530 0x7fc90010cdc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009ca0 tx=0x7fc8f802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 -- 172.21.15.12:0/27475493 shutdown_connections 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 --2- 172.21.15.12:0/27475493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900106530 0x7fc90010cdc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 --2- 172.21.15.12:0/27475493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc900105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 --2- 172.21.15.12:0/27475493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900104980 0x7fc900104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 -- 172.21.15.12:0/27475493 >> 172.21.15.12:0/27475493 conn(0x7fc9000fff10 msgr2=0x7fc900102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 -- 172.21.15.12:0/27475493 shutdown_connections 2024-04-26T21:59:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.756+0000 7fc906600640 1 -- 172.21.15.12:0/27475493 wait complete. 2024-04-26T21:59:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.757+0000 7fc906600640 1 Processor -- start 2024-04-26T21:59:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.757+0000 7fc906600640 1 -- start start 2024-04-26T21:59:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.757+0000 7fc906600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900104980 0x7fc90019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc906600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc90019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc906600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc906600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001104b0 con 0x7fc900104980 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc906600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc900110330 con 0x7fc900106530 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc906600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc900110630 con 0x7fc900105b70 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc90019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:37988/0 (socket says 172.21.15.12:37988) 2024-04-26T21:59:51.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 -- 172.21.15.12:0/348446126 learned_addr learned my addr 172.21.15.12:0/348446126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905200640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900104980 0x7fc90019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 -- 172.21.15.12:0/348446126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 msgr2=0x7fc90019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc90019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 -- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900104980 msgr2=0x7fc90019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900104980 0x7fc90019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc905c00640 1 -- 172.21.15.12:0/348446126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9001a2780 con 0x7fc900106530 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.758+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc90019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc905c00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80097b0 tx=0x7fc8f8031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc8fd600640 1 -- 172.21.15.12:0/348446126 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8003b70 con 0x7fc900106530 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc8fd600640 1 -- 172.21.15.12:0/348446126 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f8003d10 con 0x7fc900106530 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001a2a10 con 0x7fc900106530 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc8fd600640 1 -- 172.21.15.12:0/348446126 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f802fc10 con 0x7fc900106530 2024-04-26T21:59:51.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.759+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a2f60 con 0x7fc900106530 2024-04-26T21:59:51.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.764+0000 7fc90ab6b740 1 Processor -- start 2024-04-26T21:59:51.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.764+0000 7fc8fd600640 1 -- 172.21.15.12:0/348446126 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc8f8004040 con 0x7fc900106530 2024-04-26T21:59:51.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.764+0000 7fc90ab6b740 1 -- start start 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.764+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af3be20 0x55f44af3e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f44ae9a600 con 0x55f44af362c0 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f44ae37100 con 0x55f44af36cb0 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f44ae965c0 con 0x55f44af3be20 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:37990/0 (socket says 172.21.15.12:37990) 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34136/0 (socket says 172.21.15.12:34136) 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 -- 172.21.15.12:0/210276738 learned_addr learned my addr 172.21.15.12:0/210276738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af3be20 0x55f44af3e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 -- 172.21.15.12:0/210276738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af3be20 msgr2=0x55f44af3e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af3be20 0x55f44af3e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 -- 172.21.15.12:0/210276738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 msgr2=0x55f44af366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc905200640 1 -- 172.21.15.12:0/210276738 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44abac2d0 con 0x55f44af36cb0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.765+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc905200640 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ca0 tx=0x7fc8f4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5300f03379312520 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8fd600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 0x7fc8e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8f2000640 1 -- 172.21.15.12:0/210276738 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4004220 con 0x55f44af36cb0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8f2000640 1 -- 172.21.15.12:0/210276738 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f40043c0 con 0x55f44af36cb0 2024-04-26T21:59:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8f2000640 1 -- 172.21.15.12:0/210276738 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4019810 con 0x55f44af36cb0 2024-04-26T21:59:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8fd600640 1 -- 172.21.15.12:0/348446126 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8f80f1b80 con 0x7fc900106530 2024-04-26T21:59:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.766+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 0x7fc8e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- 172.21.15.12:0/210276738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 msgr2=0x55f44af3b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ca0 tx=0x7fc8f4011810 comp rx=0 tx=0).stop 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- 172.21.15.12:0/210276738 shutdown_connections 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af3be20 0x55f44af3e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af3b7b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/210276738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af362c0 0x55f44af366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- 172.21.15.12:0/210276738 >> 172.21.15.12:0/210276738 conn(0x55f44aecad10 msgr2=0x55f44af31d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 0x7fc8e80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009710 tx=0x7fc8ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- 172.21.15.12:0/210276738 shutdown_connections 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- 172.21.15.12:0/210276738 wait complete. 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.767+0000 7fc90ab6b740 1 -- start start 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 Processor -- start 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 0x55f44af406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af36cb0 0x55f44af40bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f44af3edd0 con 0x55f44af3be20 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f44ae21f70 con 0x55f44af362c0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f44af3ef50 con 0x55f44af36cb0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc90ab6b740 10 client.0 _ll_register_callbacks cb 0x55f44ae90480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.768+0000 7fc905c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 0x55f44af406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34144/0 (socket says 172.21.15.12:34144) 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 -- 172.21.15.12:0/3984441049 learned_addr learned my addr 172.21.15.12:0/3984441049 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af36cb0 0x55f44af40bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 -- 172.21.15.12:0/3984441049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af36cb0 msgr2=0x55f44af40bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af36cb0 0x55f44af40bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 -- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 msgr2=0x55f44af406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 0x55f44af406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 -- 172.21.15.12:0/3984441049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44ac766b0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905200640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 0x55f44af406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc905c00640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80ba020 tx=0x7fc8f80bd700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7e54d375ee2729 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc8e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.769+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80d7060 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8f80bbec0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f40043c0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc8f80be2c0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc90ab6b740 10 client.14586 fetch_fsmap learned FSMap version 17 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f44af3edd0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f40043c0 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fc8f80de070 con 0x55f44af3be20 2024-04-26T21:59:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.770+0000 7fc90ab6b740 10 client.14586 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc90ab6b740 10 client.14586 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc90ab6b740 20 client.14586 populate_metadata read hostname 'smithi012' 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44af4d3d0 0x55f44af4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc90ab6b740 4 client.14586 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f44aea8210 con 0x55f44af4d3d0 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.771+0000 7fc905200640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44af4d3d0 0x55f44af4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.772+0000 7fc905200640 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44af4d3d0 0x55f44af4e840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b6df3abd1b8328 server_cookie=95407beea4114314 in_seq=0 out_seq=0 2024-04-26T21:59:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.772+0000 7fc8e6a00640 10 client.14586 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.772+0000 7fc8e6a00640 1 -- 172.21.15.12:0/3984441049 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f44aea8210 con 0x55f44af4d3d0 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.772+0000 7fc8e6a00640 10 client.14586 handle_command_reply: tid=0 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.772+0000 7fc90ab6b740 1 client.14586 shutdown 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 msgr2=0x55f44af47c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80ba020 tx=0x7fc8f80bd700 comp rx=0 tx=0).stop 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 shutdown_connections 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44af4d3d0 0x55f44af4e840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af47c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af36cb0 0x55f44af40bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/3984441049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af362c0 0x55f44af406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc8e6a00640 0 client.14586 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 >> 172.21.15.12:0/3984441049 conn(0x55f44aecad10 msgr2=0x55f44af30f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 shutdown_connections 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 1 -- 172.21.15.12:0/3984441049 wait complete. 2024-04-26T21:59:51.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.773+0000 7fc90ab6b740 20 client.14586 trim_cache size 0 max 16384 2024-04-26T21:59:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.793+0000 7fc90ab6b740 1 Processor -- start 2024-04-26T21:59:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.793+0000 7fc90ab6b740 1 -- start start 2024-04-26T21:59:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.793+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.793+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 0x55f44af47bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4cb80 con 0x55f44af36cb0 2024-04-26T21:59:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4d250 con 0x55f44af3be20 2024-04-26T21:59:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4d920 con 0x55f44af362c0 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:37994/0 (socket says 172.21.15.12:37994) 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 0x55f44af47bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 -- 172.21.15.12:0/39270321 learned_addr learned my addr 172.21.15.12:0/39270321 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 -- 172.21.15.12:0/39270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 msgr2=0x55f44af4bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af4bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 -- 172.21.15.12:0/39270321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 msgr2=0x55f44af47bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 0x55f44af47bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc905c00640 1 -- 172.21.15.12:0/39270321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44af2d680 con 0x55f44af3be20 2024-04-26T21:59:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.794+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 0x55f44af47bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc905c00640 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8f802f550 tx=0x7fc8f802fed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a1c7c32bffefe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc8f1600640 1 -- 172.21.15.12:0/39270321 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80bdbc0 con 0x55f44af3be20 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc8f1600640 1 -- 172.21.15.12:0/39270321 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f8036030 con 0x55f44af3be20 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc8f1600640 1 -- 172.21.15.12:0/39270321 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80bde70 con 0x55f44af3be20 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc90ab6b740 1 -- 172.21.15.12:0/39270321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 msgr2=0x55f44af4c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.795+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8f802f550 tx=0x7fc8f802fed0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 -- 172.21.15.12:0/39270321 shutdown_connections 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af3be20 0x55f44af4c2d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af36cb0 0x55f44af47bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/39270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 -- 172.21.15.12:0/39270321 >> 172.21.15.12:0/39270321 conn(0x55f44aecad10 msgr2=0x55f44af2cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 -- 172.21.15.12:0/39270321 shutdown_connections 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 -- 172.21.15.12:0/39270321 wait complete. 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 -- start start 2024-04-26T21:59:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.796+0000 7fc90ab6b740 1 Processor -- start 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af43560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4e6c0 con 0x55f44af3be20 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4dff0 con 0x55f44af36cb0 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f44af4ed90 con 0x55f44af362c0 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc90ab6b740 10 client.0 _ll_register_callbacks cb 0x7fc8f8003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc905c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc905200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af43560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc8ffe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38002/0 (socket says 172.21.15.12:38002) 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc8ffe00640 1 -- 172.21.15.12:0/4286064168 learned_addr learned my addr 172.21.15.12:0/4286064168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc905c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34170/0 (socket says 172.21.15.12:34170) 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.797+0000 7fc8ffe00640 1 -- 172.21.15.12:0/4286064168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 msgr2=0x55f44af43560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af43560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8ffe00640 1 -- 172.21.15.12:0/4286064168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 msgr2=0x55f44af46ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8ffe00640 1 -- 172.21.15.12:0/4286064168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44af470b0 con 0x55f44af36cb0 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc905200640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc905c00640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8ffe00640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec004560 tx=0x7fc8ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6a1f8c7ad5d823 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec003d70 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f8036030 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8ec02c480 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc8ec005020 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc90ab6b740 10 client.24695 fetch_fsmap learned FSMap version 17 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.798+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f44afbbeb0 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f8036030 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fc8ec02c680 con 0x55f44af36cb0 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 10 client.24695 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 10 client.24695 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 20 client.24695 populate_metadata read hostname 'smithi012' 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44afc0b40 0x55f44afbf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 4 client.24695 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:59:51.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "status"}) -- 0x55f44aea2080 con 0x55f44afc0b40 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.799+0000 7fc905200640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44afc0b40 0x55f44afbf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc905200640 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44afc0b40 0x55f44afbf850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1642b48a91ab6805 server_cookie=e1b430ce73133f32 in_seq=0 out_seq=0 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc8e6a00640 10 client.24695 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc8e6a00640 1 -- 172.21.15.12:0/4286064168 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f44aea2080 con 0x55f44afc0b40 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc8e6a00640 10 client.24695 handle_command_reply: tid=0 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 client.24695 shutdown 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 msgr2=0x55f44af43aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec004560 tx=0x7fc8ec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 shutdown_connections 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f44afc0b40 0x55f44afbf850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f44af3be20 0x55f44af46ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f44af36cb0 0x55f44af43aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 --2- 172.21.15.12:0/4286064168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f44af362c0 0x55f44af43560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 >> 172.21.15.12:0/4286064168 conn(0x55f44aecad10 msgr2=0x55f44af4bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.800+0000 7fc8e6a00640 0 client.24695 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 shutdown_connections 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc90ab6b740 1 -- 172.21.15.12:0/4286064168 wait complete. 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc90ab6b740 20 client.24695 trim_cache size 0 max 16384 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24581, 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-26T21:59:51.805 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 7.7305291570000003, 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 17, 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 14.203582516000001 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T21:59:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 msgr2=0x7fc8e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 0x7fc8e80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009710 tx=0x7fc8ec005850 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 msgr2=0x7fc9001a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80097b0 tx=0x7fc8f8031770 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 shutdown_connections 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc8e80a3bb0 0x7fc8e80a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc900106530 0x7fc9001a2110 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc900105b70 0x7fc90019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 --2- 172.21.15.12:0/348446126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc900104980 0x7fc90019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.801+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 >> 172.21.15.12:0/348446126 conn(0x7fc9000fff10 msgr2=0x7fc900108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.802+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 shutdown_connections 2024-04-26T21:59:51.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:51.802+0000 7fc906600640 1 -- 172.21.15.12:0/348446126 wait complete. 2024-04-26T21:59:51.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:59:51.819 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:59:51.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:59:52.285 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:59:52.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:52 smithi154 ceph-mon[32578]: pgmap v324: 129 pgs: 13 creating+peering, 2 unknown, 114 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-26T21:59:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:52 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2199091019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:52.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:52 smithi012 ceph-mon[28184]: pgmap v324: 129 pgs: 13 creating+peering, 2 unknown, 114 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-26T21:59:52.440 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:52 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2199091019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:52.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:52 smithi077 ceph-mon[32280]: pgmap v324: 129 pgs: 13 creating+peering, 2 unknown, 114 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-26T21:59:52.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:52 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2199091019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:54.272 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:54 smithi012 ceph-mon[28184]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-26T21:59:54.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:54 smithi154 ceph-mon[32578]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-26T21:59:54.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:54 smithi077 ceph-mon[32280]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-04-26T21:59:54.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.546+0000 7f9f1c400700 1 -- 172.21.15.12:0/3769802108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 msgr2=0x7f9f14112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.546+0000 7f9f1c400700 1 --2- 172.21.15.12:0/3769802108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f14112510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b0a0 tx=0x7f9f0c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 -- 172.21.15.12:0/3769802108 shutdown_connections 2024-04-26T21:59:54.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 --2- 172.21.15.12:0/3769802108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f14112510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 --2- 172.21.15.12:0/3769802108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f1410b640 0x7f9f1410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 --2- 172.21.15.12:0/3769802108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f14105640 0x7f9f14105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 -- 172.21.15.12:0/3769802108 >> 172.21.15.12:0/3769802108 conn(0x7f9f14101130 msgr2=0x7f9f14103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 -- 172.21.15.12:0/3769802108 shutdown_connections 2024-04-26T21:59:54.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.547+0000 7f9f1c400700 1 -- 172.21.15.12:0/3769802108 wait complete. 2024-04-26T21:59:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.548+0000 7f9f1c400700 1 Processor -- start 2024-04-26T21:59:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.548+0000 7f9f1c400700 1 -- start start 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f141a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 0x7f9f1419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f141158d0 con 0x7f9f14106000 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14115750 con 0x7f9f1410b640 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1c400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14115a50 con 0x7f9f14105640 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:48896/0 (socket says 172.21.15.12:48896) 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 -- 172.21.15.12:0/2443016473 learned_addr learned my addr 172.21.15.12:0/2443016473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19000700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f141a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f1a400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 0x7f9f1419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 -- 172.21.15.12:0/2443016473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 msgr2=0x7f9f1419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 0x7f9f1419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 -- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 msgr2=0x7f9f141a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.549+0000 7f9f19a00700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f141a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f19a00700 1 -- 172.21.15.12:0/2443016473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1419c230 con 0x7f9f14105640 2024-04-26T21:59:54.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f1a400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 0x7f9f1419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:54.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f19a00700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e9a0 tx=0x7f9f0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800ca10 con 0x7f9f14105640 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f0800cbb0 con 0x7f9f14105640 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08018680 con 0x7f9f14105640 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1419c520 con 0x7f9f14105640 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.550+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f141a87e0 con 0x7f9f14105640 2024-04-26T21:59:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.551+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1406a110 con 0x7f9f14105640 2024-04-26T21:59:54.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.553+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f08010430 con 0x7f9f14105640 2024-04-26T21:59:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.555+0000 7f9f12000700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 0x7f9efc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.555+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9f080d0780 con 0x7f9f14105640 2024-04-26T21:59:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.555+0000 7f9f19000700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 0x7f9efc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.556+0000 7f9f19000700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 0x7f9efc0a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f1410ac00 tx=0x7f9f00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.559+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f0809cec0 con 0x7f9f14105640 2024-04-26T21:59:54.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.843+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f1402e490 con 0x7f9f14105640 2024-04-26T21:59:54.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.844+0000 7f9f12000700 1 -- 172.21.15.12:0/2443016473 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f9f0809c7c0 con 0x7f9f14105640 2024-04-26T21:59:54.844 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.848+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 msgr2=0x7f9efc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.848+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 0x7f9efc0a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f1410ac00 tx=0x7f9f00009260 comp rx=0 tx=0).stop 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.848+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 msgr2=0x7f9f141a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.848+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e9a0 tx=0x7f9f0800ecb0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 shutdown_connections 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1410b640 0x7f9f1419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f9efc0a3c40 0x7f9efc0a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f14105640 0x7f9f141a1910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 --2- 172.21.15.12:0/2443016473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f14106000 0x7f9f141a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 >> 172.21.15.12:0/2443016473 conn(0x7f9f14101130 msgr2=0x7f9f14101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 shutdown_connections 2024-04-26T21:59:54.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:54.849+0000 7f9f1c400700 1 -- 172.21.15.12:0/2443016473 wait complete. 2024-04-26T21:59:55.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:55 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2443016473' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:59:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:55 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2443016473' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:59:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:55 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2443016473' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:59:55.661 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:59:55.661 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:59:55.661 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-04-26T21:59:55.661 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T21:59:55.662 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:59:55.662 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:59:55.662 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:59:55.662 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:59:55.662 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T21:59:55.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:55.662 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:59:55.725 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T21:59:55.726 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-04-26T21:59:55.727 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:59:55.727 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:59:55.727 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T21:59:55.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:59:55.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-04-26T21:59:55.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-04-26T21:59:55.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:59:55.728 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T21:59:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:59:56.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:56.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:56.234 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:59:56.304 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-04-26T21:59:56.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:56.308 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-04-26T21:59:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-04-26T21:59:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:59:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:59:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:59:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:59:56.368 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T21:59:56.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:59:56.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:56 smithi077 ceph-mon[32280]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T21:59:56.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:56 smithi012 ceph-mon[28184]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T21:59:56.542 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:56 smithi154 ceph-mon[32578]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T21:59:57.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:57.128 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-26T21:59:57.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:57.189 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-04-26T21:59:57.245 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:59:57.245 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T21:59:57.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:57.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T21:59:57.245 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T21:59:57.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:59:57.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:57.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T21:59:57.449 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:59:57.449 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T21:59:57.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T21:59:57.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:59:57.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T21:59:57.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T21:59:57.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T21:59:57.450 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T21:59:57.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:59:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:57.759 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:59:57.759 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T21:59:57.759 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-04-26T21:59:57.759 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:59:57.759 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T21:59:57.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:59:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:57.973 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T21:59:57.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:59:57.973 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:59:58.020 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T21:59:58.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:59:58.020 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T21:59:58.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-26T21:59:58.143 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:59:58.194 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-26T21:59:58.197 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-26T21:59:58.197 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-04-26T21:59:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-26T21:59:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a6bb1b6c637214b26c63eb8d3a7c9700cef3bff1a34c1d95b6a87d3ad880aaae/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/af89d1bf7d1c69e7efbbbf83300349d8d46ed172df801eb87541fac17fcc277a/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/7fa021415b50f1031a32497a648a30314cedc25ee2199a23e462ec223cbfae4b/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a381dafc1a6615a96b26723db7efb2cbd08bb4159c3c74f69bee6c7df4f1262f/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c762e82db22dc4019ff033966c5ae5c165de4351e1680aa159bb15451e4a3fde/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ad086e2a11b5e354c3af3960da2e01b071fa8f906ad1a04d201f1f2ad19d7a6/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4414b9eb6a3bda5886dc5a1c2b0f5a5a6c28287fdaf1518f475e39a06a329962/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3fda39eab39a46cbd9390310fe25c67f70b2c0bd3bd967071c4f0327a5e0974d/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c9117712ae1b6122edfedf07f2bf4a5641394d3e473fa28a3e94544c082d02f4/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e242cf000a15577ed5729d5dc8bbca13c243a3db7c4fc7dfcc0261c30656b289/merged 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:58.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:58.202 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-26T21:59:58.202 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 21:59:58 smithi012 ceph-mon[28184]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 795 B/s wr, 0 op/s 2024-04-26T21:59:58.247 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:59:58.248 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T21:59:58.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-26T21:59:58.363 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-26T21:59:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-26T21:59:58.412 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a6bb1b6c637214b26c63eb8d3a7c9700cef3bff1a34c1d95b6a87d3ad880aaae/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/af89d1bf7d1c69e7efbbbf83300349d8d46ed172df801eb87541fac17fcc277a/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/7fa021415b50f1031a32497a648a30314cedc25ee2199a23e462ec223cbfae4b/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a381dafc1a6615a96b26723db7efb2cbd08bb4159c3c74f69bee6c7df4f1262f/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c762e82db22dc4019ff033966c5ae5c165de4351e1680aa159bb15451e4a3fde/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ad086e2a11b5e354c3af3960da2e01b071fa8f906ad1a04d201f1f2ad19d7a6/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4414b9eb6a3bda5886dc5a1c2b0f5a5a6c28287fdaf1518f475e39a06a329962/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3fda39eab39a46cbd9390310fe25c67f70b2c0bd3bd967071c4f0327a5e0974d/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c9117712ae1b6122edfedf07f2bf4a5641394d3e473fa28a3e94544c082d02f4/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e242cf000a15577ed5729d5dc8bbca13c243a3db7c4fc7dfcc0261c30656b289/merged 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-26T21:59:58.413 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:58.414 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:58.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:58.415 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-26T21:59:58.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 21:59:58 smithi077 ceph-mon[32280]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 795 B/s wr, 0 op/s 2024-04-26T21:59:58.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2024-04-26T21:59:58.500+0000 7f387e4f7480 -1 init, newargv = 0x5647f7001510 newargc=15 2024-04-26T21:59:58.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[71838]: starting ceph client 2024-04-26T21:59:58.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[71838]: starting fuse 2024-04-26T21:59:58.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 21:59:58 smithi154 ceph-mon[32578]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 795 B/s wr, 0 op/s 2024-04-26T21:59:59.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-26T21:59:59.659 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:59:59.718 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-26T21:59:59.719 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-26T21:59:59.720 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:59:59.721 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a6bb1b6c637214b26c63eb8d3a7c9700cef3bff1a34c1d95b6a87d3ad880aaae/merged 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/af89d1bf7d1c69e7efbbbf83300349d8d46ed172df801eb87541fac17fcc277a/merged 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/7fa021415b50f1031a32497a648a30314cedc25ee2199a23e462ec223cbfae4b/merged 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/a381dafc1a6615a96b26723db7efb2cbd08bb4159c3c74f69bee6c7df4f1262f/merged 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c762e82db22dc4019ff033966c5ae5c165de4351e1680aa159bb15451e4a3fde/merged 2024-04-26T21:59:59.722 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ad086e2a11b5e354c3af3960da2e01b071fa8f906ad1a04d201f1f2ad19d7a6/merged 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/4414b9eb6a3bda5886dc5a1c2b0f5a5a6c28287fdaf1518f475e39a06a329962/merged 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3fda39eab39a46cbd9390310fe25c67f70b2c0bd3bd967071c4f0327a5e0974d/merged 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/c9117712ae1b6122edfedf07f2bf4a5641394d3e473fa28a3e94544c082d02f4/merged 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e242cf000a15577ed5729d5dc8bbca13c243a3db7c4fc7dfcc0261c30656b289/merged 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T21:59:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-26T21:59:59.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:59.725 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-26T21:59:59.781 INFO:teuthology.orchestra.run.smithi012.stdout:114 2024-04-26T21:59:59.781 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-26T21:59:59.782 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-04-26T21:59:59.783 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-04-26T21:59:59.784 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:59:59.784 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-26T21:59:59.784 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2024-04-26T21:59:59.784 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-04-26T21:59:59.962 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.71838.asok 2024-04-26T21:59:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T21:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:59:59.980 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71838.asok 2024-04-26T21:59:59.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:59.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71838.asok status 2024-04-26T22:00:00.227 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:00:00.227 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-04-26T22:00:00.227 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "71838", 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24715, 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-04-26T22:00:00.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 24715 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3766850597 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T22:00:00.229 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3766850597 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.24715 172.21.15.12:0/3766850597", 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "172.21.15.12:0/3766850597", 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 17, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 90, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-04-26T22:00:00.230 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:00:00.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:00 smithi012 ceph-mon[28184]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:00:00.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:00 smithi012 ceph-mon[28184]: overall HEALTH_OK 2024-04-26T22:00:00.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:00.243 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-04-26T22:00:00.243 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:00:00.297 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-04-26T22:00:00.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:00:00.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:00.297 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:00:00.380 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T22:00:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:00 smithi077 ceph-mon[32280]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:00:00.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:00 smithi077 ceph-mon[32280]: overall HEALTH_OK 2024-04-26T22:00:00.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:00.533 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2024-04-26T22:00:00.533 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T22:00:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:00.543 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:00:00.544 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:00:00.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:00.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:00 smithi154 ceph-mon[32578]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:00:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:00 smithi154 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T22:00:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 -- 172.21.15.12:0/1954228532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 msgr2=0x7fcc68102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/1954228532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc68102bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009ca0 tx=0x7fcc5802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 -- 172.21.15.12:0/1954228532 shutdown_connections 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/1954228532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc6810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/1954228532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc68103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/1954228532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc68102bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 -- 172.21.15.12:0/1954228532 >> 172.21.15.12:0/1954228532 conn(0x7fcc680fdd40 msgr2=0x7fcc68100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 -- 172.21.15.12:0/1954228532 shutdown_connections 2024-04-26T22:00:00.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.805+0000 7fcc6ca00640 1 -- 172.21.15.12:0/1954228532 wait complete. 2024-04-26T22:00:00.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.806+0000 7fcc6ca00640 1 Processor -- start 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.806+0000 7fcc6ca00640 1 -- start start 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc6819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc6819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc681a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6810e330 con 0x7fcc681039a0 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6810e1b0 con 0x7fcc68104360 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc6ca00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6810e4b0 con 0x7fcc681027b0 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc6819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc66800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc6819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc66800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc6819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58400/0 (socket says 172.21.15.12:58400) 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc66800640 1 -- 172.21.15.12:0/66475083 learned_addr learned my addr 172.21.15.12:0/66475083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.807+0000 7fcc67200640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc681a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 -- 172.21.15.12:0/66475083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 msgr2=0x7fcc6819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc6819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 -- 172.21.15.12:0/66475083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 msgr2=0x7fcc681a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc681a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 -- 172.21.15.12:0/66475083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc681a2770 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc67200640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc681a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc65e00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc6819b090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e9f0 tx=0x7fcc5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400ccc0 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc540108e0 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc681a2a60 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.808+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54018880 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.809+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc681a2fb0 con 0x7fcc681039a0 2024-04-26T22:00:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.810+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc681a31e0 con 0x7fcc681039a0 2024-04-26T22:00:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.813+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc54010430 con 0x7fcc681039a0 2024-04-26T22:00:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.814+0000 7fcc5f400640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 0x7fcc4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.814+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcc540d0e20 con 0x7fcc681039a0 2024-04-26T22:00:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.814+0000 7fcc66800640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 0x7fcc4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.815+0000 7fcc66800640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 0x7fcc4c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fe0 tx=0x7fcc58005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:00.816+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcc5409cf10 con 0x7fcc681039a0 2024-04-26T22:00:01.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.023+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc681a3570 con 0x7fcc681039a0 2024-04-26T22:00:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.028+0000 7fcc5f400640 1 -- 172.21.15.12:0/66475083 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7fcc681a3570 con 0x7fcc681039a0 2024-04-26T22:00:01.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:01.030 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 msgr2=0x7fcc4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 0x7fcc4c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fe0 tx=0x7fcc58005f70 comp rx=0 tx=0).stop 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 msgr2=0x7fcc6819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc6819b090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e9f0 tx=0x7fcc5400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 shutdown_connections 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcc4c0a3bb0 0x7fcc4c0a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc68104360 0x7fcc681a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc681039a0 0x7fcc6819b090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 --2- 172.21.15.12:0/66475083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc681027b0 0x7fcc6819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 >> 172.21.15.12:0/66475083 conn(0x7fcc680fdd40 msgr2=0x7fcc68074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 shutdown_connections 2024-04-26T22:00:01.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.031+0000 7fcc6ca00640 1 -- 172.21.15.12:0/66475083 wait complete. 2024-04-26T22:00:01.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:00:01.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 -- 172.21.15.12:0/3679096919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc104b90 msgr2=0x7f8ccc104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3679096919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc104b90 0x7f8ccc104fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009900 tx=0x7f8cbc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 -- 172.21.15.12:0/3679096919 shutdown_connections 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3679096919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3679096919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc105d80 0x7f8ccc106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3679096919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc104b90 0x7f8ccc104fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 -- 172.21.15.12:0/3679096919 >> 172.21.15.12:0/3679096919 conn(0x7f8ccc100120 msgr2=0x7f8ccc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 -- 172.21.15.12:0/3679096919 shutdown_connections 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.272+0000 7f8cd1000640 1 -- 172.21.15.12:0/3679096919 wait complete. 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 Processor -- start 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 -- start start 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc104b90 0x7f8ccc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1106e0 con 0x7f8ccc105d80 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc110560 con 0x7f8ccc106740 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.273+0000 7f8cd1000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc110860 con 0x7f8ccc104b90 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36856/0 (socket says 172.21.15.12:36856) 2024-04-26T22:00:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 -- 172.21.15.12:0/3293529602 learned_addr learned my addr 172.21.15.12:0/3293529602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cca800640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc9e00640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc104b90 0x7f8ccc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 -- 172.21.15.12:0/3293529602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc104b90 msgr2=0x7f8ccc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc104b90 0x7f8ccc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 -- 172.21.15.12:0/3293529602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 msgr2=0x7f8ccc1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cc1400640 1 -- 172.21.15.12:0/3293529602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc1a2a30 con 0x7f8ccc105d80 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.274+0000 7f8cca800640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cc1400640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400d950 tx=0x7f8cb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4017030 con 0x7f8ccc105d80 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb4004020 con 0x7f8ccc105d80 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1a2d20 con 0x7f8ccc105d80 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40047a0 con 0x7f8ccc105d80 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.275+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a3260 con 0x7f8ccc105d80 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.276+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc10bff0 con 0x7f8ccc105d80 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.279+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8cb4009a30 con 0x7f8ccc105d80 2024-04-26T22:00:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.280+0000 7f8cc3400640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 0x7f8cb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.280+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8cb40d06c0 con 0x7f8ccc105d80 2024-04-26T22:00:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.280+0000 7f8cc9e00640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 0x7f8cb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.281+0000 7f8cc9e00640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 0x7f8cb00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005ab0 tx=0x7f8cbc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.283+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8cb409ce80 con 0x7f8ccc105d80 2024-04-26T22:00:01.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:01 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/66475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:01 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/66475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:01.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.491+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ccc02dd20 con 0x7f8ccc105d80 2024-04-26T22:00:01.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.492+0000 7f8cc3400640 1 -- 172.21.15.12:0/3293529602 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f8cb409c780 con 0x7f8ccc105d80 2024-04-26T22:00:01.493 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.494+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 msgr2=0x7f8cb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.494+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 0x7f8cb00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005ab0 tx=0x7f8cbc005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.494+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 msgr2=0x7f8ccc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.494+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400d950 tx=0x7f8cb400de10 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 shutdown_connections 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8cb00a3c30 0x7f8cb00a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ccc106740 0x7f8ccc1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ccc105d80 0x7f8ccc19b350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 --2- 172.21.15.12:0/3293529602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc104b90 0x7f8ccc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 >> 172.21.15.12:0/3293529602 conn(0x7f8ccc100120 msgr2=0x7f8ccc1091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 shutdown_connections 2024-04-26T22:00:01.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.495+0000 7f8cd1000640 1 -- 172.21.15.12:0/3293529602 wait complete. 2024-04-26T22:00:01.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:00:01.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:01 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/66475083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 -- 172.21.15.12:0/4248041259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274410a980 msgr2=0x7f27441112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 --2- 172.21.15.12:0/4248041259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274410a980 0x7f27441112a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f273c009860 tx=0x7f273c02a340 comp rx=0 tx=0).stop 2024-04-26T22:00:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 -- 172.21.15.12:0/4248041259 shutdown_connections 2024-04-26T22:00:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 --2- 172.21.15.12:0/4248041259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274410a980 0x7f27441112a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 --2- 172.21.15.12:0/4248041259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2744109fc0 0x7f274410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 --2- 172.21.15.12:0/4248041259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744108e70 0x7f2744109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 -- 172.21.15.12:0/4248041259 >> 172.21.15.12:0/4248041259 conn(0x7f2744077e00 msgr2=0x7f274407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 -- 172.21.15.12:0/4248041259 shutdown_connections 2024-04-26T22:00:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.732+0000 7f2749a00640 1 -- 172.21.15.12:0/4248041259 wait complete. 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.733+0000 7f2749a00640 1 Processor -- start 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.733+0000 7f2749a00640 1 -- start start 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.733+0000 7f2749a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744109fc0 0x7f274419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274410a980 0x7f27441a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27441149a0 con 0x7f274410a980 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2744114820 con 0x7f2744108e70 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2744114b20 con 0x7f2744109fc0 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2749000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274410a980 0x7f27441a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58208/0 (socket says 172.21.15.12:58208) 2024-04-26T22:00:01.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 -- 172.21.15.12:0/4016674378 learned_addr learned my addr 172.21.15.12:0/4016674378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743400640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744109fc0 0x7f274419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 -- 172.21.15.12:0/4016674378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744109fc0 msgr2=0x7f274419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744109fc0 0x7f274419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 -- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274410a980 msgr2=0x7f27441a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274410a980 0x7f27441a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.734+0000 7f2743e00640 1 -- 172.21.15.12:0/4016674378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441a6c60 con 0x7f2744108e70 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2743e00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f273400c420 tx=0x7f2734009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734003970 con 0x7f2744108e70 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2734004020 con 0x7f2744108e70 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273400b980 con 0x7f2744108e70 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441a6f50 con 0x7f2744108e70 2024-04-26T22:00:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.735+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a7410 con 0x7f2744108e70 2024-04-26T22:00:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.736+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744110120 con 0x7f2744108e70 2024-04-26T22:00:01.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.738+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f273401d020 con 0x7f2744108e70 2024-04-26T22:00:01.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.739+0000 7f2740c00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 0x7f27280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.740+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f27340d0460 con 0x7f2744108e70 2024-04-26T22:00:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.740+0000 7f2743400640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 0x7f27280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.740+0000 7f2743400640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 0x7f27280a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f27441a0550 tx=0x7f272c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.744+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f273409cba0 con 0x7f2744108e70 2024-04-26T22:00:01.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.948+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2744063560 con 0x7f2744108e70 2024-04-26T22:00:01.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.949+0000 7f2740c00640 1 -- 172.21.15.12:0/4016674378 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f273409c4a0 con 0x7f2744108e70 2024-04-26T22:00:01.950 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:00:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 msgr2=0x7f27280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 0x7f27280a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f27441a0550 tx=0x7f272c005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 msgr2=0x7f274419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f273400c420 tx=0x7f2734009f90 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 shutdown_connections 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f27280a3c30 0x7f27280a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f274410a980 0x7f27441a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2744109fc0 0x7f274419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 --2- 172.21.15.12:0/4016674378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2744108e70 0x7f274419f040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 >> 172.21.15.12:0/4016674378 conn(0x7f2744077e00 msgr2=0x7f274410d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.952+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 shutdown_connections 2024-04-26T22:00:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:01.953+0000 7f2749a00640 1 -- 172.21.15.12:0/4016674378 wait complete. 2024-04-26T22:00:01.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.189+0000 7f2beba00640 1 -- 172.21.15.12:0/4051490620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4104b70 msgr2=0x7f2be4104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.189+0000 7f2beba00640 1 --2- 172.21.15.12:0/4051490620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4104b70 0x7f2be4104f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009900 tx=0x7f2bd402a410 comp rx=0 tx=0).stop 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 -- 172.21.15.12:0/4051490620 shutdown_connections 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 --2- 172.21.15.12:0/4051490620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 0x7f2be410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 --2- 172.21.15.12:0/4051490620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4105d60 0x7f2be41061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 --2- 172.21.15.12:0/4051490620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4104b70 0x7f2be4104f90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 -- 172.21.15.12:0/4051490620 >> 172.21.15.12:0/4051490620 conn(0x7f2be4100120 msgr2=0x7f2be4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 -- 172.21.15.12:0/4051490620 shutdown_connections 2024-04-26T22:00:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.190+0000 7f2beba00640 1 -- 172.21.15.12:0/4051490620 wait complete. 2024-04-26T22:00:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 Processor -- start 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 -- start start 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 0x7f2be419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 0x7f2be41a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4110690 con 0x7f2be4105d60 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4110510 con 0x7f2be4106720 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2beba00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4110810 con 0x7f2be4104b70 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2be9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2be3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 0x7f2be419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2be9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58448/0 (socket says 172.21.15.12:58448) 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.191+0000 7f2be9000640 1 -- 172.21.15.12:0/621365829 learned_addr learned my addr 172.21.15.12:0/621365829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be9a00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 0x7f2be41a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 -- 172.21.15.12:0/621365829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 msgr2=0x7f2be419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 -- 172.21.15.12:0/621365829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 msgr2=0x7f2be41a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 0x7f2be41a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 -- 172.21.15.12:0/621365829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be41a2a10 con 0x7f2be4105d60 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be9000640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.192+0000 7f2be3e00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 0x7f2be419b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e9f0 tx=0x7f2bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.193+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000ccc0 con 0x7f2be4105d60 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.193+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41a2d00 con 0x7f2be4105d60 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.193+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd0005020 con 0x7f2be4105d60 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.193+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004a00 con 0x7f2be4105d60 2024-04-26T22:00:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.193+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a31f0 con 0x7f2be4105d60 2024-04-26T22:00:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.194+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be410c000 con 0x7f2be4105d60 2024-04-26T22:00:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.197+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bd0004ba0 con 0x7f2be4105d60 2024-04-26T22:00:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.198+0000 7f2be1600640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 0x7f2bc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.198+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bd00d0500 con 0x7f2be4105d60 2024-04-26T22:00:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.199+0000 7f2be9000640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 0x7f2bc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.200+0000 7f2be9000640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 0x7f2bc80a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4032fd0 tx=0x7f2bd40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.203+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2bd009cca0 con 0x7f2be4105d60 2024-04-26T22:00:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:02 smithi012 ceph-mon[28184]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:02 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3293529602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:02 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4016674378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.412+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2be40635d0 con 0x7f2be4105d60 2024-04-26T22:00:02.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.413+0000 7f2be1600640 1 -- 172.21.15.12:0/621365829 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f2bd009c5a0 con 0x7f2be4105d60 2024-04-26T22:00:02.414 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 msgr2=0x7f2bc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 0x7f2bc80a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4032fd0 tx=0x7f2bd40314a0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 msgr2=0x7f2be419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 0x7f2be419b330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e9f0 tx=0x7f2bd000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 shutdown_connections 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f2bc80a3c30 0x7f2bc80a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2be4106720 0x7f2be41a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2be4105d60 0x7f2be419b330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 --2- 172.21.15.12:0/621365829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2be4104b70 0x7f2be419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 >> 172.21.15.12:0/621365829 conn(0x7f2be4100120 msgr2=0x7f2be41091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 shutdown_connections 2024-04-26T22:00:02.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.416+0000 7f2beba00640 1 -- 172.21.15.12:0/621365829 wait complete. 2024-04-26T22:00:02.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:00:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:02 smithi077 ceph-mon[32280]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:02 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3293529602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:02 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4016674378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:02 smithi154 ceph-mon[32578]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:02.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:02 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3293529602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:02 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4016674378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:02.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 -- 172.21.15.12:0/3022163941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10a180 msgr2=0x7fe64c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 --2- 172.21.15.12:0/3022163941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10a180 0x7fe64c10a600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe638009900 tx=0x7fe63802a400 comp rx=0 tx=0).stop 2024-04-26T22:00:02.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 -- 172.21.15.12:0/3022163941 shutdown_connections 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 --2- 172.21.15.12:0/3022163941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c10ab40 0x7fe64c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 --2- 172.21.15.12:0/3022163941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10a180 0x7fe64c10a600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 --2- 172.21.15.12:0/3022163941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c108f90 0x7fe64c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 -- 172.21.15.12:0/3022163941 >> 172.21.15.12:0/3022163941 conn(0x7fe64c077f30 msgr2=0x7fe64c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 -- 172.21.15.12:0/3022163941 shutdown_connections 2024-04-26T22:00:02.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.658+0000 7fe651400640 1 -- 172.21.15.12:0/3022163941 wait complete. 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.659+0000 7fe651400640 1 Processor -- start 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 -- start start 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 0x7fe64c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10ab40 0x7fe64c1a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c114ad0 con 0x7fe64c10ab40 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c114950 con 0x7fe64c108f90 2024-04-26T22:00:02.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe651400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c114c50 con 0x7fe64c10a180 2024-04-26T22:00:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe64b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10ab40 0x7fe64c1a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe649e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe649e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58470/0 (socket says 172.21.15.12:58470) 2024-04-26T22:00:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe649e00640 1 -- 172.21.15.12:0/1834607626 learned_addr learned my addr 172.21.15.12:0/1834607626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.660+0000 7fe64a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 0x7fe64c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 -- 172.21.15.12:0/1834607626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 msgr2=0x7fe64c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 -- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10ab40 msgr2=0x7fe64c1a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10ab40 0x7fe64c1a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 -- 172.21.15.12:0/1834607626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1a6db0 con 0x7fe64c108f90 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe649e00640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe64a800640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 0x7fe64c19f190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00e9c0 tx=0x7fe63c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00ccc0 con 0x7fe64c108f90 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c0035f0 con 0x7fe64c108f90 2024-04-26T22:00:02.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00ce60 con 0x7fe64c108f90 2024-04-26T22:00:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.661+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1a70a0 con 0x7fe64c108f90 2024-04-26T22:00:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.662+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1a7570 con 0x7fe64c108f90 2024-04-26T22:00:02.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.663+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c110380 con 0x7fe64c108f90 2024-04-26T22:00:02.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.665+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe63c004000 con 0x7fe64c108f90 2024-04-26T22:00:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.666+0000 7fe643400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 0x7fe6300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.666+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe63c0d0a00 con 0x7fe64c108f90 2024-04-26T22:00:02.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.666+0000 7fe649e00640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 0x7fe6300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.667+0000 7fe649e00640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 0x7fe6300a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe638009230 tx=0x7fe6380058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.670+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe63c09d140 con 0x7fe64c108f90 2024-04-26T22:00:02.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.880+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe64c063560 con 0x7fe64c108f90 2024-04-26T22:00:02.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.881+0000 7fe643400640 1 -- 172.21.15.12:0/1834607626 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fe63c09ca40 con 0x7fe64c108f90 2024-04-26T22:00:02.882 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:00:02.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 msgr2=0x7fe6300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 0x7fe6300a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe638009230 tx=0x7fe6380058d0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 msgr2=0x7fe64c19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 0x7fe64c19f190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00e9c0 tx=0x7fe63c00ee80 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 shutdown_connections 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe6300a3c30 0x7fe6300a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe64c10ab40 0x7fe64c1a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe64c10a180 0x7fe64c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 --2- 172.21.15.12:0/1834607626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c108f90 0x7fe64c19f190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 >> 172.21.15.12:0/1834607626 conn(0x7fe64c077f30 msgr2=0x7fe64c10d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.884+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 shutdown_connections 2024-04-26T22:00:02.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:02.885+0000 7fe651400640 1 -- 172.21.15.12:0/1834607626 wait complete. 2024-04-26T22:00:02.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:00:03.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 -- 172.21.15.12:0/573969433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 msgr2=0x7f8a84107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 --2- 172.21.15.12:0/573969433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a84107f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009900 tx=0x7f8a7002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 -- 172.21.15.12:0/573969433 shutdown_connections 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 --2- 172.21.15.12:0/573969433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a8410a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 --2- 172.21.15.12:0/573969433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a84107f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 --2- 172.21.15.12:0/573969433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a841055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.115+0000 7f8a88a00640 1 -- 172.21.15.12:0/573969433 >> 172.21.15.12:0/573969433 conn(0x7f8a840fcbf0 msgr2=0x7f8a840ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.116+0000 7f8a88a00640 1 -- 172.21.15.12:0/573969433 shutdown_connections 2024-04-26T22:00:03.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.116+0000 7f8a88a00640 1 -- 172.21.15.12:0/573969433 wait complete. 2024-04-26T22:00:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 Processor -- start 2024-04-26T22:00:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 -- start start 2024-04-26T22:00:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a8419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a841a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410e280 con 0x7f8a841031e0 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410e100 con 0x7f8a84105b10 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a88a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410e400 con 0x7f8a84108440 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.117+0000 7f8a83400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36910/0 (socket says 172.21.15.12:36910) 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 -- 172.21.15.12:0/3283924166 learned_addr learned my addr 172.21.15.12:0/3283924166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83e00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a841a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a82a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a8419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 -- 172.21.15.12:0/3283924166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 msgr2=0x7f8a841a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a841a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 -- 172.21.15.12:0/3283924166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 msgr2=0x7f8a8419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a8419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83400640 1 -- 172.21.15.12:0/3283924166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a841a2800 con 0x7f8a841031e0 2024-04-26T22:00:03.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.118+0000 7f8a83e00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a841a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a83400640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea60 tx=0x7f8a7400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400cd20 con 0x7f8a841031e0 2024-04-26T22:00:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a740035f0 con 0x7f8a841031e0 2024-04-26T22:00:03.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841a2af0 con 0x7f8a841031e0 2024-04-26T22:00:03.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74010690 con 0x7f8a841031e0 2024-04-26T22:00:03.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.119+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a841a3010 con 0x7f8a841031e0 2024-04-26T22:00:03.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.120+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84109f90 con 0x7f8a841031e0 2024-04-26T22:00:03.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.122+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a740037a0 con 0x7f8a841031e0 2024-04-26T22:00:03.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.124+0000 7f8a7be00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 0x7f8a680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.124+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8a740d09a0 con 0x7f8a841031e0 2024-04-26T22:00:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.124+0000 7f8a82a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 0x7f8a680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.125+0000 7f8a82a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 0x7f8a680a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009230 tx=0x7f8a700314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.129+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a7409d0e0 con 0x7f8a841031e0 2024-04-26T22:00:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.352+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a840635d0 con 0x7f8a841031e0 2024-04-26T22:00:03.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.353+0000 7f8a7be00640 1 -- 172.21.15.12:0/3283924166 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f8a7409c9e0 con 0x7f8a841031e0 2024-04-26T22:00:03.354 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:03.354 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:03.354 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3243088974 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1310858189 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3387900079 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/600788188 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2638555806 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4197187136 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:03.355 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-04-26T22:00:03.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 msgr2=0x7f8a680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 0x7f8a680a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009230 tx=0x7f8a700314a0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 msgr2=0x7f8a8419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea60 tx=0x7f8a7400ee60 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 shutdown_connections 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8a680a3bb0 0x7f8a680a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a84108440 0x7f8a841a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a84105b10 0x7f8a8419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 --2- 172.21.15.12:0/3283924166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a841031e0 0x7f8a8419abe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.356+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 >> 172.21.15.12:0/3283924166 conn(0x7f8a840fcbf0 msgr2=0x7f8a84102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.357+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 shutdown_connections 2024-04-26T22:00:03.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.357+0000 7f8a88a00640 1 -- 172.21.15.12:0/3283924166 wait complete. 2024-04-26T22:00:03.376 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:00:03.376 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:00:03.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:03.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:03 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/621365829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:03 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1834607626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:03 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/621365829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:03 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1834607626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:03 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/621365829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:03 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1834607626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 -- 172.21.15.12:0/97518989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 msgr2=0x7f5e64104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 --2- 172.21.15.12:0/97518989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e64104dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a410 comp rx=0 tx=0).stop 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 -- 172.21.15.12:0/97518989 shutdown_connections 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 --2- 172.21.15.12:0/97518989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64106560 0x7f5e6410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 --2- 172.21.15.12:0/97518989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64105ba0 0x7f5e64106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 --2- 172.21.15.12:0/97518989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e64104dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 -- 172.21.15.12:0/97518989 >> 172.21.15.12:0/97518989 conn(0x7f5e640fff40 msgr2=0x7f5e64102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 -- 172.21.15.12:0/97518989 shutdown_connections 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.646+0000 7f5e6b200640 1 -- 172.21.15.12:0/97518989 wait complete. 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.647+0000 7f5e6b200640 1 Processor -- start 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 -- start start 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64105ba0 0x7f5e6419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64106560 0x7f5e641a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64110500 con 0x7f5e641049b0 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64110380 con 0x7f5e64106560 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64110680 con 0x7f5e64105ba0 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e69e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e69e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36924/0 (socket says 172.21.15.12:36924) 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e69e00640 1 -- 172.21.15.12:0/747052824 learned_addr learned my addr 172.21.15.12:0/747052824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e69400640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64105ba0 0x7f5e6419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.648+0000 7f5e6a800640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64106560 0x7f5e641a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 -- 172.21.15.12:0/747052824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64105ba0 msgr2=0x7f5e6419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64105ba0 0x7f5e6419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 -- 172.21.15.12:0/747052824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64106560 msgr2=0x7f5e641a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64106560 0x7f5e641a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 -- 172.21.15.12:0/747052824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641a2790 con 0x7f5e641049b0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e69e00640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e58031500 tx=0x7f5e58003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5803f070 con 0x7f5e641049b0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e58031c50 con 0x7f5e641049b0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035030 con 0x7f5e641049b0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.649+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641a2a20 con 0x7f5e641049b0 2024-04-26T22:00:03.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.650+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e641a2f60 con 0x7f5e641049b0 2024-04-26T22:00:03.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.651+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6410bd20 con 0x7f5e641049b0 2024-04-26T22:00:03.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.654+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e58042070 con 0x7f5e641049b0 2024-04-26T22:00:03.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.656+0000 7f5e5ea00640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.656+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5e580f14c0 con 0x7f5e641049b0 2024-04-26T22:00:03.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.656+0000 7f5e69400640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.657+0000 7f5e69400640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e6419c080 tx=0x7f5e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.658+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e580bdc00 con 0x7f5e641049b0 2024-04-26T22:00:03.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.867+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e6402dca0 con 0x7f5e641049b0 2024-04-26T22:00:03.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.869+0000 7f5e5ea00640 1 -- 172.21.15.12:0/747052824 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f5e580bd500 con 0x7f5e641049b0 2024-04-26T22:00:03.871 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:03.871 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.872+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 msgr2=0x7f5e4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.872+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e6419c080 tx=0x7f5e54005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.872+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 msgr2=0x7f5e6419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.872+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e58031500 tx=0x7f5e58003c10 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 shutdown_connections 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5e4c0a3c30 0x7f5e4c0a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e64106560 0x7f5e641a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e64105ba0 0x7f5e6419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 --2- 172.21.15.12:0/747052824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e641049b0 0x7f5e6419ab70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 >> 172.21.15.12:0/747052824 conn(0x7f5e640fff40 msgr2=0x7f5e64108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 shutdown_connections 2024-04-26T22:00:03.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:03.873+0000 7f5e6b200640 1 -- 172.21.15.12:0/747052824 wait complete. 2024-04-26T22:00:03.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:00:03.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 -- 172.21.15.12:0/932227036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c102930 msgr2=0x7f260c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 --2- 172.21.15.12:0/932227036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c102930 0x7f260c102db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009900 tx=0x7f25f402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 -- 172.21.15.12:0/932227036 shutdown_connections 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 --2- 172.21.15.12:0/932227036 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c1032f0 0x7f260c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 --2- 172.21.15.12:0/932227036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c102930 0x7f260c102db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 --2- 172.21.15.12:0/932227036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c101740 0x7f260c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.114+0000 7f2611000640 1 -- 172.21.15.12:0/932227036 >> 172.21.15.12:0/932227036 conn(0x7f260c0fccd0 msgr2=0x7f260c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.115+0000 7f2611000640 1 -- 172.21.15.12:0/932227036 shutdown_connections 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.115+0000 7f2611000640 1 -- 172.21.15.12:0/932227036 wait complete. 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 Processor -- start 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 -- start start 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c101740 0x7f260c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c1032f0 0x7f260c1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10e3f0 con 0x7f260c101740 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10e270 con 0x7f260c1032f0 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f2611000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10e570 con 0x7f260c102930 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f260b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c101740 0x7f260c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.116+0000 7f260aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58536/0 (socket says 172.21.15.12:58536) 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 -- 172.21.15.12:0/2925020705 learned_addr learned my addr 172.21.15.12:0/2925020705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260be00640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c1032f0 0x7f260c1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 -- 172.21.15.12:0/2925020705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c1032f0 msgr2=0x7f260c1a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c1032f0 0x7f260c1a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 -- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c101740 msgr2=0x7f260c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c101740 0x7f260c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 -- 172.21.15.12:0/2925020705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c1a0680 con 0x7f260c102930 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.117+0000 7f260aa00640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f25f4005ab0 tx=0x7f25f4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.118+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f403f070 con 0x7f260c102930 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.118+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c1a0910 con 0x7f260c102930 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.118+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f4031c50 con 0x7f260c102930 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.118+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4035030 con 0x7f260c102930 2024-04-26T22:00:04.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.118+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a0e50 con 0x7f260c102930 2024-04-26T22:00:04.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.119+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260c108a60 con 0x7f260c102930 2024-04-26T22:00:04.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.121+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f25f4042070 con 0x7f260c102930 2024-04-26T22:00:04.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.123+0000 7f2603e00640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 0x7f25ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.123+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f25f40f1440 con 0x7f260c102930 2024-04-26T22:00:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.123+0000 7f260b400640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 0x7f25ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.124+0000 7f260b400640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 0x7f25ec0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f260c102790 tx=0x7f25fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.125+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f25f40bd530 con 0x7f260c102930 2024-04-26T22:00:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.339+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f260c02dad0 con 0x7f260c102930 2024-04-26T22:00:04.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.340+0000 7f2603e00640 1 -- 172.21.15.12:0/2925020705 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f25f40c1830 con 0x7f260c102930 2024-04-26T22:00:04.341 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 msgr2=0x7f25ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 0x7f25ec0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f260c102790 tx=0x7f25fc005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 msgr2=0x7f260c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f25f4005ab0 tx=0x7f25f4003b90 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 shutdown_connections 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f25ec0a3c30 0x7f25ec0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f260c1032f0 0x7f260c1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f260c102930 0x7f260c198fa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 --2- 172.21.15.12:0/2925020705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f260c101740 0x7f260c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 >> 172.21.15.12:0/2925020705 conn(0x7f260c0fccd0 msgr2=0x7f260c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 shutdown_connections 2024-04-26T22:00:04.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.343+0000 7f2611000640 1 -- 172.21.15.12:0/2925020705 wait complete. 2024-04-26T22:00:04.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:00:04.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:04 smithi012 ceph-mon[28184]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:04.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:04 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3283924166' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:04.439 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:04 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/747052824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:04.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:04 smithi077 ceph-mon[32280]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:04.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:04 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3283924166' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:04.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:04 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/747052824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:04.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 -- 172.21.15.12:0/931726445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c108520 msgr2=0x7f122c10a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 --2- 172.21.15.12:0/931726445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c108520 0x7f122c10a970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1224009900 tx=0x7f122402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 -- 172.21.15.12:0/931726445 shutdown_connections 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 --2- 172.21.15.12:0/931726445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c108520 0x7f122c10a970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 --2- 172.21.15.12:0/931726445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 0x7f122c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 --2- 172.21.15.12:0/931726445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c1032c0 0x7f122c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 -- 172.21.15.12:0/931726445 >> 172.21.15.12:0/931726445 conn(0x7f122c0fccd0 msgr2=0x7f122c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 -- 172.21.15.12:0/931726445 shutdown_connections 2024-04-26T22:00:04.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.583+0000 7f1231c00640 1 -- 172.21.15.12:0/931726445 wait complete. 2024-04-26T22:00:04.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.584+0000 7f1231c00640 1 Processor -- start 2024-04-26T22:00:04.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 -- start start 2024-04-26T22:00:04.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 0x7f122c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 0x7f122c1a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10e3c0 con 0x7f122c1032c0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10e240 con 0x7f122c105bf0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f1231c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10e540 con 0x7f122c108520 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f122be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f122be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36956/0 (socket says 172.21.15.12:36956) 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f122b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 0x7f122c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.585+0000 7f122be00640 1 -- 172.21.15.12:0/725372900 learned_addr learned my addr 172.21.15.12:0/725372900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1231200640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 0x7f122c1a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 -- 172.21.15.12:0/725372900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 msgr2=0x7f122c1a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 0x7f122c1a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 -- 172.21.15.12:0/725372900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 msgr2=0x7f122c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 0x7f122c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 -- 172.21.15.12:0/725372900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c1a2880 con 0x7f122c1032c0 2024-04-26T22:00:04.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1231200640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 0x7f122c1a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f122be00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f121c00ebc0 tx=0x7f121c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c010630 con 0x7f122c1032c0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c010c80 con 0x7f122c1032c0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c1a2b70 con 0x7f122c1032c0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.586+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c018b60 con 0x7f122c1032c0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.587+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c1a30c0 con 0x7f122c1032c0 2024-04-26T22:00:04.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.588+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c10a010 con 0x7f122c1032c0 2024-04-26T22:00:04.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.590+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f121c018d00 con 0x7f122c1032c0 2024-04-26T22:00:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.591+0000 7f1228c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 0x7f12100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.591+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f121c0d0d50 con 0x7f122c1032c0 2024-04-26T22:00:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.592+0000 7f122b400640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 0x7f12100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.592+0000 7f122b400640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 0x7f12100a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f122c06a620 tx=0x7f1214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.595+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f121c0d4030 con 0x7f122c1032c0 2024-04-26T22:00:04.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:04 smithi154 ceph-mon[32578]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T22:00:04.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:04 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3283924166' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:04.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:04 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/747052824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:04.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.802+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f122c063530 con 0x7f122c1032c0 2024-04-26T22:00:04.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.803+0000 7f1228c00640 1 -- 172.21.15.12:0/725372900 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f121c09d4f0 con 0x7f122c1032c0 2024-04-26T22:00:04.804 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:00:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 msgr2=0x7f12100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 0x7f12100a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f122c06a620 tx=0x7f1214005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 msgr2=0x7f122c19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f121c00ebc0 tx=0x7f121c00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 shutdown_connections 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f12100a3bb0 0x7f12100a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122c108520 0x7f122c1a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f122c105bf0 0x7f122c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 --2- 172.21.15.12:0/725372900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f122c1032c0 0x7f122c19ac70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.805+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 >> 172.21.15.12:0/725372900 conn(0x7f122c0fccd0 msgr2=0x7f122c102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.806+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 shutdown_connections 2024-04-26T22:00:04.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:04.806+0000 7f1231c00640 1 -- 172.21.15.12:0/725372900 wait complete. 2024-04-26T22:00:04.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.029+0000 7ff594e00640 1 -- 172.21.15.12:0/1100696978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 msgr2=0x7ff59010cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.029+0000 7ff594e00640 1 --2- 172.21.15.12:0/1100696978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff59010cac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff588009860 tx=0x7ff58802a340 comp rx=0 tx=0).stop 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 -- 172.21.15.12:0/1100696978 shutdown_connections 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 --2- 172.21.15.12:0/1100696978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff59010cac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 --2- 172.21.15.12:0/1100696978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590075390 0x7ff590075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 --2- 172.21.15.12:0/1100696978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590076970 0x7ff590074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 -- 172.21.15.12:0/1100696978 >> 172.21.15.12:0/1100696978 conn(0x7ff5900fdce0 msgr2=0x7ff590100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 -- 172.21.15.12:0/1100696978 shutdown_connections 2024-04-26T22:00:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.030+0000 7ff594e00640 1 -- 172.21.15.12:0/1100696978 wait complete. 2024-04-26T22:00:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.031+0000 7ff594e00640 1 Processor -- start 2024-04-26T22:00:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.031+0000 7ff594e00640 1 -- start start 2024-04-26T22:00:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.031+0000 7ff594e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 0x7ff59019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.031+0000 7ff594e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.031+0000 7ff594e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff5901a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff594e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff590110460 con 0x7ff590076970 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff594e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901102e0 con 0x7ff590075390 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff594e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901105e0 con 0x7ff59010a630 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36968/0 (socket says 172.21.15.12:36968) 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 -- 172.21.15.12:0/441253444 learned_addr learned my addr 172.21.15.12:0/441253444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58f400640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 0x7ff59019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58fe00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff5901a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 -- 172.21.15.12:0/441253444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 msgr2=0x7ff5901a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff5901a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 -- 172.21.15.12:0/441253444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 msgr2=0x7ff59019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 0x7ff59019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58ea00640 1 -- 172.21.15.12:0/441253444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901a2720 con 0x7ff590076970 2024-04-26T22:00:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.032+0000 7ff58f400640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 0x7ff59019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff58ea00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e9c0 tx=0x7ff57800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800cde0 con 0x7ff590076970 2024-04-26T22:00:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a2a10 con 0x7ff590076970 2024-04-26T22:00:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5780035f0 con 0x7ff590076970 2024-04-26T22:00:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578018630 con 0x7ff590076970 2024-04-26T22:00:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.033+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5901a2fb0 con 0x7ff590076970 2024-04-26T22:00:05.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.034+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff590109360 con 0x7ff590076970 2024-04-26T22:00:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.037+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff578004000 con 0x7ff590076970 2024-04-26T22:00:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.038+0000 7ff587e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 0x7ff5740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.038+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff5780d08e0 con 0x7ff590076970 2024-04-26T22:00:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.038+0000 7ff58f400640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 0x7ff5740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.039+0000 7ff58f400640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 0x7ff5740a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff580005fd0 tx=0x7ff580009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.042+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff57809d020 con 0x7ff590076970 2024-04-26T22:00:05.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.247+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff590063530 con 0x7ff590076970 2024-04-26T22:00:05.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.248+0000 7ff587e00640 1 -- 172.21.15.12:0/441253444 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff578009510 con 0x7ff590076970 2024-04-26T22:00:05.249 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.250+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 msgr2=0x7ff5740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.250+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 0x7ff5740a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff580005fd0 tx=0x7ff580009500 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.250+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 msgr2=0x7ff59019b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.250+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e9c0 tx=0x7ff57800ee80 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 shutdown_connections 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5740a3bb0 0x7ff5740a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff59010a630 0x7ff5901a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff590076970 0x7ff59019b040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 --2- 172.21.15.12:0/441253444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff590075390 0x7ff59019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 >> 172.21.15.12:0/441253444 conn(0x7ff5900fdce0 msgr2=0x7ff590106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 shutdown_connections 2024-04-26T22:00:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.251+0000 7ff594e00640 1 -- 172.21.15.12:0/441253444 wait complete. 2024-04-26T22:00:05.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:00:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 -- 172.21.15.12:0/778993548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204105bf0 msgr2=0x7f2204107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 --2- 172.21.15.12:0/778993548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204105bf0 0x7f2204107fe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 -- 172.21.15.12:0/778993548 shutdown_connections 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 --2- 172.21.15.12:0/778993548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2204108520 0x7f220410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 --2- 172.21.15.12:0/778993548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204105bf0 0x7f2204107fe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 --2- 172.21.15.12:0/778993548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22041032c0 0x7f22041056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 -- 172.21.15.12:0/778993548 >> 172.21.15.12:0/778993548 conn(0x7f22040fccd0 msgr2=0x7f22040ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 -- 172.21.15.12:0/778993548 shutdown_connections 2024-04-26T22:00:05.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.470+0000 7f2203e00640 1 -- 172.21.15.12:0/778993548 wait complete. 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.471+0000 7f2203e00640 1 Processor -- start 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.471+0000 7f2203e00640 1 -- start start 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2204105bf0 0x7f220419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204108520 0x7f22041a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f220410e3b0 con 0x7f2204108520 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f220410e230 con 0x7f22041032c0 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f220410e530 con 0x7f2204105bf0 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f21fa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2204105bf0 0x7f220419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2203400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204108520 0x7f22041a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58340/0 (socket says 172.21.15.12:58340) 2024-04-26T22:00:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 -- 172.21.15.12:0/147718442 learned_addr learned my addr 172.21.15.12:0/147718442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 -- 172.21.15.12:0/147718442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2204105bf0 msgr2=0x7f220419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2204105bf0 0x7f220419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 -- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204108520 msgr2=0x7f22041a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.472+0000 7f2202a00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204108520 0x7f22041a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f2202a00640 1 -- 172.21.15.12:0/147718442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22041a28f0 con 0x7f22041032c0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f2202a00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f21f400e9f0 tx=0x7f21f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f400cd80 con 0x7f22041032c0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041a2be0 con 0x7f22041032c0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f40108e0 con 0x7f22041032c0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4018980 con 0x7f22041032c0 2024-04-26T22:00:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.473+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041a3150 con 0x7f22041032c0 2024-04-26T22:00:05.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.475+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220410a070 con 0x7f22041032c0 2024-04-26T22:00:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.476+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f21f4010430 con 0x7f22041032c0 2024-04-26T22:00:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.478+0000 7f21fbe00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 0x7f21e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.478+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f21f40d0970 con 0x7f22041032c0 2024-04-26T22:00:05.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.478+0000 7f21fa000640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 0x7f21e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.479+0000 7f21fa000640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 0x7f21e80a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005ab0 tx=0x7f21ec005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:05 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2925020705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:05 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/725372900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.482+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f21f409ca60 con 0x7f22041032c0 2024-04-26T22:00:05.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:05 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2925020705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:05 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/725372900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:05 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2925020705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:05 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/725372900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:00:05.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.679+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2204063530 con 0x7f22041032c0 2024-04-26T22:00:05.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.680+0000 7f21fbe00640 1 -- 172.21.15.12:0/147718442 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f21f40a0d60 con 0x7f22041032c0 2024-04-26T22:00:05.681 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:00:05.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 msgr2=0x7f21e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 0x7f21e80a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005ab0 tx=0x7f21ec005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:05.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 msgr2=0x7f220419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f21f400e9f0 tx=0x7f21f400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 shutdown_connections 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f21e80a3bb0 0x7f21e80a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2204108520 0x7f22041a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2204105bf0 0x7f220419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 --2- 172.21.15.12:0/147718442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22041032c0 0x7f220419acb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 >> 172.21.15.12:0/147718442 conn(0x7f22040fccd0 msgr2=0x7f2204102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.682+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 shutdown_connections 2024-04-26T22:00:05.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.683+0000 7f2203e00640 1 -- 172.21.15.12:0/147718442 wait complete. 2024-04-26T22:00:05.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.902+0000 7fe073800640 1 -- 172.21.15.12:0/2698691084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c105da0 msgr2=0x7fe06c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.902+0000 7fe073800640 1 --2- 172.21.15.12:0/2698691084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c105da0 0x7fe06c106220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe054009900 tx=0x7fe05402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 -- 172.21.15.12:0/2698691084 shutdown_connections 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 --2- 172.21.15.12:0/2698691084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c106760 0x7fe06c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 --2- 172.21.15.12:0/2698691084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c105da0 0x7fe06c106220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 --2- 172.21.15.12:0/2698691084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c104bb0 0x7fe06c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 -- 172.21.15.12:0/2698691084 >> 172.21.15.12:0/2698691084 conn(0x7fe06c100120 msgr2=0x7fe06c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 -- 172.21.15.12:0/2698691084 shutdown_connections 2024-04-26T22:00:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.903+0000 7fe073800640 1 -- 172.21.15.12:0/2698691084 wait complete. 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 Processor -- start 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 -- start start 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 0x7fe06c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 0x7fe06c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 0x7fe06c1a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c1106c0 con 0x7fe06c106760 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c110540 con 0x7fe06c105da0 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.904+0000 7fe073800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c110840 con 0x7fe06c104bb0 2024-04-26T22:00:05.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe071800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 0x7fe06c1a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe06be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 0x7fe06c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 0x7fe06c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe06be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 0x7fe06c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58346/0 (socket says 172.21.15.12:58346) 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe071800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 0x7fe06c1a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37010/0 (socket says 172.21.15.12:37010) 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe071800640 1 -- 172.21.15.12:0/1269361916 learned_addr learned my addr 172.21.15.12:0/1269361916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 -- 172.21.15.12:0/1269361916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 msgr2=0x7fe06c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 0x7fe06c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 -- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 msgr2=0x7fe06c1a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 0x7fe06c1a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 -- 172.21.15.12:0/1269361916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c1a2a00 con 0x7fe06c104bb0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.905+0000 7fe070e00640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 0x7fe06c19ae30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.906+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c010630 con 0x7fe06c104bb0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.906+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c1a2cf0 con 0x7fe06c104bb0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.906+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1a31b0 con 0x7fe06c104bb0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.906+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c010c80 con 0x7fe06c104bb0 2024-04-26T22:00:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.906+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c018c50 con 0x7fe06c104bb0 2024-04-26T22:00:05.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.907+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe06c10c010 con 0x7fe06c104bb0 2024-04-26T22:00:05.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.909+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe05c01f020 con 0x7fe06c104bb0 2024-04-26T22:00:05.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.911+0000 7fe069600640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.911+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe05c0d0de0 con 0x7fe06c104bb0 2024-04-26T22:00:05.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.911+0000 7fe06be00640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.912+0000 7fe06be00640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 0x7fe0500a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe054000c00 tx=0x7fe0540314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:05.913+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe05c09d520 con 0x7fe06c104bb0 2024-04-26T22:00:06.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.123+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe06c02dad0 con 0x7fe06c104bb0 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.124+0000 7fe069600640 1 -- 172.21.15.12:0/1269361916 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fe05c09ce20 con 0x7fe06c104bb0 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3243088974 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1310858189 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:06.125 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3387900079 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/600788188 2024-04-27T21:51:07.889475+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2638555806 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4197187136 2024-04-27T21:51:42.978579+0000 2024-04-26T22:00:06.126 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 msgr2=0x7fe0500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 0x7fe0500a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe054000c00 tx=0x7fe0540314a0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 msgr2=0x7fe06c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 0x7fe06c19ae30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 shutdown_connections 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe06c106760 0x7fe06c1a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe06c105da0 0x7fe06c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 --2- 172.21.15.12:0/1269361916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe06c104bb0 0x7fe06c19ae30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 >> 172.21.15.12:0/1269361916 conn(0x7fe06c100120 msgr2=0x7fe06c1091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 shutdown_connections 2024-04-26T22:00:06.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.127+0000 7fe073800640 1 -- 172.21.15.12:0/1269361916 wait complete. 2024-04-26T22:00:06.145 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-04-26T22:00:06.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:06.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T22:00:06.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[71838]: fuse finished with error 0 and tester_r 0 2024-04-26T22:00:06.235 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:00:06.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:00:06.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:06.298 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T22:00:06.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:06.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:06 smithi012 ceph-mon[28184]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:06.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:06 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/441253444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:06 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/147718442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:06 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1269361916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:06.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:06 smithi077 ceph-mon[32280]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:06.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:06 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/441253444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:06 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/147718442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:06 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1269361916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 -- 172.21.15.12:0/2537839452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 msgr2=0x7f4578102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 --2- 172.21.15.12:0/2537839452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f4578102bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f455c009860 tx=0x7f455c02a340 comp rx=0 tx=0).stop 2024-04-26T22:00:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 -- 172.21.15.12:0/2537839452 shutdown_connections 2024-04-26T22:00:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 --2- 172.21.15.12:0/2537839452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4578104360 0x7f457810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 --2- 172.21.15.12:0/2537839452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45781039a0 0x7f4578103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 --2- 172.21.15.12:0/2537839452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f4578102bd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 -- 172.21.15.12:0/2537839452 >> 172.21.15.12:0/2537839452 conn(0x7f45780fdd40 msgr2=0x7f4578100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 -- 172.21.15.12:0/2537839452 shutdown_connections 2024-04-26T22:00:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.576+0000 7f457e400640 1 -- 172.21.15.12:0/2537839452 wait complete. 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.577+0000 7f457e400640 1 Processor -- start 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.577+0000 7f457e400640 1 -- start start 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 0x7f457819ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f457810e2b0 con 0x7f45781039a0 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f457810e130 con 0x7f45781027b0 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457e400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f457810e430 con 0x7f4578104360 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457d000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37026/0 (socket says 172.21.15.12:37026) 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 0x7f457819ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 -- 172.21.15.12:0/636763091 learned_addr learned my addr 172.21.15.12:0/636763091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f457d000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58358/0 (socket says 172.21.15.12:58358) 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 -- 172.21.15.12:0/636763091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 msgr2=0x7f457819ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 0x7f457819ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 -- 172.21.15.12:0/636763091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 msgr2=0x7f45781989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.578+0000 7f456fe00640 1 -- 172.21.15.12:0/636763091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45781a05d0 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f457da00640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 0x7f457819ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f457d000640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f456fe00640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400cd80 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640108e0 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45781a08c0 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564018980 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.579+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45781a0e60 con 0x7f45781039a0 2024-04-26T22:00:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.581+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4578109af0 con 0x7f45781039a0 2024-04-26T22:00:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.583+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4564010430 con 0x7f45781039a0 2024-04-26T22:00:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.586+0000 7f4576000640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 0x7f45540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.586+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45640d0de0 con 0x7f45781039a0 2024-04-26T22:00:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.586+0000 7f457d000640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 0x7f45540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.587+0000 7f457d000640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 0x7f45540a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f455c02a930 tx=0x7f455c030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.588+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f456409ced0 con 0x7f45781039a0 2024-04-26T22:00:06.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:06 smithi154 ceph-mon[32578]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:06.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:06 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/441253444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:06 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/147718442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:00:06.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:06 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1269361916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:00:06.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.790+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f457802dca0 con 0x7f45781039a0 2024-04-26T22:00:06.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.792+0000 7f4576000640 1 -- 172.21.15.12:0/636763091 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f45640a11d0 con 0x7f45781039a0 2024-04-26T22:00:06.793 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:06.793 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 msgr2=0x7f45540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 0x7f45540a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f455c02a930 tx=0x7f455c030470 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 msgr2=0x7f4578198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 shutdown_connections 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f45540a3c30 0x7f45540a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4578104360 0x7f457819ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45781039a0 0x7f4578198ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 --2- 172.21.15.12:0/636763091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45781027b0 0x7f45781989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.794+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 >> 172.21.15.12:0/636763091 conn(0x7f45780fdd40 msgr2=0x7f4578106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.795+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 shutdown_connections 2024-04-26T22:00:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:06.795+0000 7f457e400640 1 -- 172.21.15.12:0/636763091 wait complete. 2024-04-26T22:00:06.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.019+0000 7fe63de00640 1 -- 172.21.15.12:0/844802328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 msgr2=0x7fe6381113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.019+0000 7fe63de00640 1 --2- 172.21.15.12:0/844802328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381113d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe630009ca0 tx=0x7fe63002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.019+0000 7fe63de00640 1 -- 172.21.15.12:0/844802328 shutdown_connections 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 --2- 172.21.15.12:0/844802328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381113d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 --2- 172.21.15.12:0/844802328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe63810a180 0x7fe63810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 --2- 172.21.15.12:0/844802328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe638108f90 0x7fe6381093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 -- 172.21.15.12:0/844802328 >> 172.21.15.12:0/844802328 conn(0x7fe638077f40 msgr2=0x7fe63807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 -- 172.21.15.12:0/844802328 shutdown_connections 2024-04-26T22:00:07.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.020+0000 7fe63de00640 1 -- 172.21.15.12:0/844802328 wait complete. 2024-04-26T22:00:07.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 Processor -- start 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 -- start start 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe638108f90 0x7fe63819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63810a180 0x7fe63819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe638114a90 con 0x7fe63810ab40 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe638114910 con 0x7fe63810a180 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.021+0000 7fe63de00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe638114c10 con 0x7fe638108f90 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37040/0 (socket says 172.21.15.12:37040) 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 -- 172.21.15.12:0/1515956407 learned_addr learned my addr 172.21.15.12:0/1515956407 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe636a00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe638108f90 0x7fe63819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe62fe00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63810a180 0x7fe63819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 -- 172.21.15.12:0/1515956407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe638108f90 msgr2=0x7fe63819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe638108f90 0x7fe63819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 -- 172.21.15.12:0/1515956407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63810a180 msgr2=0x7fe63819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63810a180 0x7fe63819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 -- 172.21.15.12:0/1515956407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6381a6d20 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.022+0000 7fe637400640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6300097b0 tx=0x7fe630031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.023+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630003b70 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.023+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe630003d10 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.023+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6381a6fb0 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.023+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63002fc10 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.023+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6381a7470 con 0x7fe63810ab40 2024-04-26T22:00:07.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.024+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6381102b0 con 0x7fe63810ab40 2024-04-26T22:00:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.026+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe630004040 con 0x7fe63810ab40 2024-04-26T22:00:07.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.027+0000 7fe63ca00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 0x7fe61c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.027+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6300f1b80 con 0x7fe63810ab40 2024-04-26T22:00:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.028+0000 7fe636a00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 0x7fe61c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.028+0000 7fe636a00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 0x7fe61c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6280095f0 tx=0x7fe628005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.031+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe6300be3e0 con 0x7fe63810ab40 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.243+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6380635d0 con 0x7fe63810ab40 2024-04-26T22:00:07.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.246+0000 7fe63ca00640 1 -- 172.21.15.12:0/1515956407 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7fe6300bdce0 con 0x7fe63810ab40 2024-04-26T22:00:07.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:07.247 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 msgr2=0x7fe61c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 0x7fe61c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6280095f0 tx=0x7fe628005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 msgr2=0x7fe6381a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6300097b0 tx=0x7fe630031770 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 shutdown_connections 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe61c0a3bb0 0x7fe61c0a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe63810ab40 0x7fe6381a66b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63810a180 0x7fe63819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 --2- 172.21.15.12:0/1515956407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe638108f90 0x7fe63819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.248+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 >> 172.21.15.12:0/1515956407 conn(0x7fe638077f40 msgr2=0x7fe63810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.249+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 shutdown_connections 2024-04-26T22:00:07.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.249+0000 7fe63de00640 1 -- 172.21.15.12:0/1515956407 wait complete. 2024-04-26T22:00:07.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T22:00:07.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:07 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/636763091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:07.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 -- 172.21.15.12:0/867363411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c101730 msgr2=0x7fa08c101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 --2- 172.21.15.12:0/867363411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c101730 0x7fa08c101b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 -- 172.21.15.12:0/867363411 shutdown_connections 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 --2- 172.21.15.12:0/867363411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c1032e0 0x7fa08c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 --2- 172.21.15.12:0/867363411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 0x7fa08c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 --2- 172.21.15.12:0/867363411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c101730 0x7fa08c101b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 -- 172.21.15.12:0/867363411 >> 172.21.15.12:0/867363411 conn(0x7fa08c0fcca0 msgr2=0x7fa08c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 -- 172.21.15.12:0/867363411 shutdown_connections 2024-04-26T22:00:07.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.488+0000 7fa08b200640 1 -- 172.21.15.12:0/867363411 wait complete. 2024-04-26T22:00:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.489+0000 7fa08b200640 1 Processor -- start 2024-04-26T22:00:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 -- start start 2024-04-26T22:00:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c101730 0x7fa08c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 0x7fa08c198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10e3e0 con 0x7fa08c1032e0 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10e260 con 0x7fa08c102920 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10e560 con 0x7fa08c101730 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37050/0 (socket says 172.21.15.12:37050) 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa08a800640 1 -- 172.21.15.12:0/2400086538 learned_addr learned my addr 172.21.15.12:0/2400086538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa089e00640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c101730 0x7fa08c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.490+0000 7fa083e00640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 0x7fa08c198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 -- 172.21.15.12:0/2400086538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c101730 msgr2=0x7fa08c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c101730 0x7fa08c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 -- 172.21.15.12:0/2400086538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 msgr2=0x7fa08c198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 0x7fa08c198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 -- 172.21.15.12:0/2400086538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c1a0690 con 0x7fa08c1032e0 2024-04-26T22:00:07.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08a800640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa08400eb10 tx=0x7fa08400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084004b20 con 0x7fa08c1032e0 2024-04-26T22:00:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c1a0980 con 0x7fa08c1032e0 2024-04-26T22:00:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa084004cc0 con 0x7fa08c1032e0 2024-04-26T22:00:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084010690 con 0x7fa08c1032e0 2024-04-26T22:00:07.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.491+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a0ed0 con 0x7fa08c1032e0 2024-04-26T22:00:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.493+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c108aa0 con 0x7fa08c1032e0 2024-04-26T22:00:07.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.495+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa084005020 con 0x7fa08c1032e0 2024-04-26T22:00:07.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.497+0000 7fa082a00640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 0x7fa0700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.497+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa0840d0be0 con 0x7fa08c1032e0 2024-04-26T22:00:07.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.497+0000 7fa089e00640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 0x7fa0700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.498+0000 7fa089e00640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 0x7fa0700a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005ab0 tx=0x7fa07c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.501+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa08409d320 con 0x7fa08c1032e0 2024-04-26T22:00:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:07 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/636763091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:07 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/636763091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:07.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:07.747+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa08c02dad0 con 0x7fa08c1032e0 2024-04-26T22:00:08.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.214+0000 7fa082a00640 1 -- 172.21.15.12:0/2400086538 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fa084087020 con 0x7fa08c1032e0 2024-04-26T22:00:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 msgr2=0x7fa0700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 0x7fa0700a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005ab0 tx=0x7fa07c005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:08.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 msgr2=0x7fa08c1a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa08400eb10 tx=0x7fa08400c2d0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 shutdown_connections 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0700a3bb0 0x7fa0700a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa08c1032e0 0x7fa08c1a0020 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa08c102920 0x7fa08c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 --2- 172.21.15.12:0/2400086538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa08c101730 0x7fa08c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 >> 172.21.15.12:0/2400086538 conn(0x7fa08c0fcca0 msgr2=0x7fa08c105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:08.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 shutdown_connections 2024-04-26T22:00:08.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.218+0000 7fa08b200640 1 -- 172.21.15.12:0/2400086538 wait complete. 2024-04-26T22:00:08.240 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T22:00:08.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 -- 172.21.15.12:0/2308440635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 msgr2=0x7f80281112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 --2- 172.21.15.12:0/2308440635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281112a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8020009ca0 tx=0x7f802002a8b0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 -- 172.21.15.12:0/2308440635 shutdown_connections 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 --2- 172.21.15.12:0/2308440635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281112a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 --2- 172.21.15.12:0/2308440635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 0x7f802810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 --2- 172.21.15.12:0/2308440635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f8028109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 -- 172.21.15.12:0/2308440635 >> 172.21.15.12:0/2308440635 conn(0x7f8028077e00 msgr2=0x7f802807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 -- 172.21.15.12:0/2308440635 shutdown_connections 2024-04-26T22:00:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.464+0000 7f8027200640 1 -- 172.21.15.12:0/2308440635 wait complete. 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.465+0000 7f8027200640 1 Processor -- start 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.465+0000 7f8027200640 1 -- start start 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.465+0000 7f8027200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.465+0000 7f8027200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 0x7f802819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.465+0000 7f8027200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8027200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80281149f0 con 0x7f8028109fc0 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8027200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114870 con 0x7f802810a980 2024-04-26T22:00:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8027200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114b70 con 0x7f8028108e70 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8026800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 0x7f802819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:58676/0 (socket says 172.21.15.12:58676) 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8026800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58400/0 (socket says 172.21.15.12:58400) 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 -- 172.21.15.12:0/1097946766 learned_addr learned my addr 172.21.15.12:0/1097946766 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 -- 172.21.15.12:0/1097946766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 msgr2=0x7f80281a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 -- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 msgr2=0x7f802819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 0x7f802819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8025e00640 1 -- 172.21.15.12:0/1097946766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80281a6bd0 con 0x7f8028108e70 2024-04-26T22:00:08.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.466+0000 7f8026800640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:08.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f8025e00640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f801800d950 tx=0x7f801800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:08.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017070 con 0x7f8028108e70 2024-04-26T22:00:08.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8018009c30 con 0x7f8028108e70 2024-04-26T22:00:08.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a6ec0 con 0x7f8028108e70 2024-04-26T22:00:08.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801800f620 con 0x7f8028108e70 2024-04-26T22:00:08.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.467+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a7380 con 0x7f8028108e70 2024-04-26T22:00:08.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.468+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80281100b0 con 0x7f8028108e70 2024-04-26T22:00:08.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.470+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8018010460 con 0x7f8028108e70 2024-04-26T22:00:08.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.472+0000 7f801ea00640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 0x7f800c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.472+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f80180d01a0 con 0x7f8028108e70 2024-04-26T22:00:08.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.472+0000 7f8025400640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 0x7f800c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.473+0000 7f8025400640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 0x7f800c0a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80281a04c0 tx=0x7f8014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:08.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.476+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8018009de0 con 0x7f8028108e70 2024-04-26T22:00:08.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:08 smithi077 ceph-mon[32280]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:08.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:08 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1515956407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:08.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:08 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:00:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:08 smithi012 ceph-mon[28184]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:08 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1515956407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:08 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:00:08.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:08 smithi154 ceph-mon[32578]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:08.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:08 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1515956407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:08.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:08 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T22:00:08.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:08.727+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f8028063560 con 0x7f8028108e70 2024-04-26T22:00:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.230+0000 7f801ea00640 1 -- 172.21.15.12:0/1097946766 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f801809c290 con 0x7f8028108e70 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 msgr2=0x7f800c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 0x7f800c0a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80281a04c0 tx=0x7f8014005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 msgr2=0x7f802819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f801800d950 tx=0x7f801800de10 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 shutdown_connections 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f800c0a3c30 0x7f800c0a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f802810a980 0x7f80281a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028109fc0 0x7f802819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 --2- 172.21.15.12:0/1097946766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028108e70 0x7f802819efb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 >> 172.21.15.12:0/1097946766 conn(0x7f8028077e00 msgr2=0x7f802810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:09.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 shutdown_connections 2024-04-26T22:00:09.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.234+0000 7f8027200640 1 -- 172.21.15.12:0/1097946766 wait complete. 2024-04-26T22:00:09.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 -- 172.21.15.12:0/2815932657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc10a6d0 msgr2=0x7f61bc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2815932657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc10a6d0 0x7f61bc10cb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009ca0 tx=0x7f61b402a940 comp rx=0 tx=0).stop 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 -- 172.21.15.12:0/2815932657 shutdown_connections 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2815932657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc10a6d0 0x7f61bc10cb70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2815932657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc075330 0x7f61bc075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2815932657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc076910 0x7f61bc074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 -- 172.21.15.12:0/2815932657 >> 172.21.15.12:0/2815932657 conn(0x7f61bc0fddd0 msgr2=0x7f61bc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 -- 172.21.15.12:0/2815932657 shutdown_connections 2024-04-26T22:00:09.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.473+0000 7f61c2e00640 1 -- 172.21.15.12:0/2815932657 wait complete. 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.474+0000 7f61c2e00640 1 Processor -- start 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.474+0000 7f61c2e00640 1 -- start start 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.474+0000 7f61c2e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075330 0x7f61bc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.474+0000 7f61c2e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.474+0000 7f61c2e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 0x7f61bc1a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c2e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1105a0 con 0x7f61bc075330 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c2e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc110420 con 0x7f61bc076910 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c2e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc110720 con 0x7f61bc10a6d0 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075330 0x7f61bc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:57996/0 (socket says 172.21.15.12:57996) 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 -- 172.21.15.12:0/2181731220 learned_addr learned my addr 172.21.15.12:0/2181731220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c2400640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 0x7f61bc1a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 -- 172.21.15.12:0/2181731220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 msgr2=0x7f61bc1a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 0x7f61bc1a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 -- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075330 msgr2=0x7f61bc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075330 0x7f61bc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c1000640 1 -- 172.21.15.12:0/2181731220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc1a2800 con 0x7f61bc076910 2024-04-26T22:00:09.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.475+0000 7f61c2400640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 0x7f61bc1a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61c1000640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61a800d950 tx=0x7f61a800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8017070 con 0x7f61bc076910 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a8009c30 con 0x7f61bc076910 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a800f620 con 0x7f61bc076910 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc1a2af0 con 0x7f61bc076910 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.476+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a2ff0 con 0x7f61bc076910 2024-04-26T22:00:09.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.477+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc109490 con 0x7f61bc076910 2024-04-26T22:00:09.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.479+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f61a8010460 con 0x7f61bc076910 2024-04-26T22:00:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.480+0000 7f61ba000640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 0x7f61a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.481+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f61a80d0250 con 0x7f61bc076910 2024-04-26T22:00:09.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.481+0000 7f61c1a00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 0x7f61a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:09 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:00:09.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:09 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:09.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:09 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1097946766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:09 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.481+0000 7f61c1a00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 0x7f61a40a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009bd0 tx=0x7f61b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:09.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.485+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f61a809c990 con 0x7f61bc076910 2024-04-26T22:00:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:09 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:00:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:09 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:09 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1097946766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:09 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:09 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2400086538' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T22:00:09.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:09 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:09.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:09 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1097946766' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:09 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T22:00:09.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:09.733+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f61bc02dad0 con 0x7f61bc076910 2024-04-26T22:00:10.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.244+0000 7f61ba000640 1 -- 172.21.15.12:0/2181731220 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f61a8009de0 con 0x7f61bc076910 2024-04-26T22:00:10.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 msgr2=0x7f61a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 0x7f61a40a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009bd0 tx=0x7f61b0005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:10.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 msgr2=0x7f61bc19b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61a800d950 tx=0x7f61a800de10 comp rx=0 tx=0).stop 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 shutdown_connections 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f61a40a3c30 0x7f61a40a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61bc10a6d0 0x7f61bc1a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61bc076910 0x7f61bc19b120 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 --2- 172.21.15.12:0/2181731220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075330 0x7f61bc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.248+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 >> 172.21.15.12:0/2181731220 conn(0x7f61bc0fddd0 msgr2=0x7f61bc106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.249+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 shutdown_connections 2024-04-26T22:00:10.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.249+0000 7f61c2e00640 1 -- 172.21.15.12:0/2181731220 wait complete. 2024-04-26T22:00:10.278 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T22:00:10.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T22:00:10.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:10.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:00:10.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:10.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:10.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2181731220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:10 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.498+0000 7fbb10a00640 1 -- 172.21.15.12:0/3860344296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 msgr2=0x7fbb0c1113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.498+0000 7fbb10a00640 1 --2- 172.21.15.12:0/3860344296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1113d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb0402a400 comp rx=0 tx=0).stop 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 -- 172.21.15.12:0/3860344296 shutdown_connections 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 --2- 172.21.15.12:0/3860344296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1113d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 --2- 172.21.15.12:0/3860344296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 0x7fbb0c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 --2- 172.21.15.12:0/3860344296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 -- 172.21.15.12:0/3860344296 >> 172.21.15.12:0/3860344296 conn(0x7fbb0c077f30 msgr2=0x7fbb0c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 -- 172.21.15.12:0/3860344296 shutdown_connections 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.499+0000 7fbb10a00640 1 -- 172.21.15.12:0/3860344296 wait complete. 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 Processor -- start 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 -- start start 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 0x7fbb0c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c114b20 con 0x7fbb0c10ab40 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1149a0 con 0x7fbb0c108f90 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.500+0000 7fbb10a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c114ca0 con 0x7fbb0c10a180 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb0a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb0a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1a66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43544/0 (socket says 172.21.15.12:43544) 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb0a800640 1 -- 172.21.15.12:0/1160537598 learned_addr learned my addr 172.21.15.12:0/1160537598 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09400640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 0x7fbb0c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 -- 172.21.15.12:0/1160537598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 msgr2=0x7fbb0c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 0x7fbb0c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 -- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 msgr2=0x7fbb0c1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 -- 172.21.15.12:0/1160537598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c1a6d60 con 0x7fbb0c108f90 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.501+0000 7fbb09e00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c19f140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9c0 tx=0x7fbafc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.502+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00cd80 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.502+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc0108e0 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.502+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c1a6ff0 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.502+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc018900 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.502+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a74b0 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.503+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c1102b0 con 0x7fbb0c108f90 2024-04-26T22:00:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.505+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbafc01f020 con 0x7fbb0c108f90 2024-04-26T22:00:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.506+0000 7fbb02a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 0x7fbaf00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.506+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbafc0d05e0 con 0x7fbb0c108f90 2024-04-26T22:00:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.506+0000 7fbb09400640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 0x7fbaf00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.507+0000 7fbb09400640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 0x7fbaf00a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c1a0650 tx=0x7fbaf8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.510+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbafc09cd20 con 0x7fbb0c108f90 2024-04-26T22:00:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:00:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:10.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2181731220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T22:00:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T22:00:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T22:00:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2181731220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:10 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T22:00:10.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.745+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbb0c062d70 con 0x7fbaf00a3bb0 2024-04-26T22:00:10.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.772+0000 7fbb02a00640 1 -- 172.21.15.12:0/1160537598 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbb0c062d70 con 0x7fbaf00a3bb0 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 msgr2=0x7fbaf00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 0x7fbaf00a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c1a0650 tx=0x7fbaf8009500 comp rx=0 tx=0).stop 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 msgr2=0x7fbb0c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c19f140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00e9c0 tx=0x7fbafc00ee80 comp rx=0 tx=0).stop 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 shutdown_connections 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbaf00a3bb0 0x7fbaf00a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb0c10ab40 0x7fbb0c1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb0c10a180 0x7fbb0c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 --2- 172.21.15.12:0/1160537598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb0c108f90 0x7fbb0c19f140 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 >> 172.21.15.12:0/1160537598 conn(0x7fbb0c077f30 msgr2=0x7fbb0c10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 shutdown_connections 2024-04-26T22:00:10.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.775+0000 7fbb10a00640 1 -- 172.21.15.12:0/1160537598 wait complete. 2024-04-26T22:00:10.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T22:00:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 -- 172.21.15.12:0/1049285910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354101c70 msgr2=0x7fa3541020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 --2- 172.21.15.12:0/1049285910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354101c70 0x7fa3541020f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009900 tx=0x7fa33c02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 -- 172.21.15.12:0/1049285910 shutdown_connections 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 --2- 172.21.15.12:0/1049285910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 0x7fa354108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 --2- 172.21.15.12:0/1049285910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354101c70 0x7fa3541020f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 --2- 172.21.15.12:0/1049285910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354100a80 0x7fa354100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 -- 172.21.15.12:0/1049285910 >> 172.21.15.12:0/1049285910 conn(0x7fa3540fbff0 msgr2=0x7fa3540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 -- 172.21.15.12:0/1049285910 shutdown_connections 2024-04-26T22:00:10.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.993+0000 7fa359c00640 1 -- 172.21.15.12:0/1049285910 wait complete. 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.994+0000 7fa359c00640 1 Processor -- start 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.994+0000 7fa359c00640 1 -- start start 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 0x7fa354104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 0x7fa354103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 0x7fa354103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410c1a0 con 0x7fa354101c70 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410c020 con 0x7fa354100a80 2024-04-26T22:00:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410c320 con 0x7fa354102630 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa353400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 0x7fa354103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa353400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 0x7fa354103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43564/0 (socket says 172.21.15.12:43564) 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa353400640 1 -- 172.21.15.12:0/1693505811 learned_addr learned my addr 172.21.15.12:0/1693505811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa353e00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 0x7fa354104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.995+0000 7fa359200640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 0x7fa354103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 -- 172.21.15.12:0/1693505811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 msgr2=0x7fa354103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 0x7fa354103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 -- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 msgr2=0x7fa354103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 0x7fa354103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 -- 172.21.15.12:0/1693505811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354104010 con 0x7fa354100a80 2024-04-26T22:00:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa353e00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 0x7fa354104c00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa3440097d0 tx=0x7fa344009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344018030 con 0x7fa354100a80 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34400e6b0 con 0x7fa354100a80 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344003850 con 0x7fa354100a80 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.996+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3541a2c00 con 0x7fa354100a80 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.997+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541a3150 con 0x7fa354100a80 2024-04-26T22:00:11.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:10.998+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3541a3320 con 0x7fa354100a80 2024-04-26T22:00:11.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.000+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa34401b070 con 0x7fa354100a80 2024-04-26T22:00:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.001+0000 7fa350c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 0x7fa3380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.001+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa3440d0460 con 0x7fa354100a80 2024-04-26T22:00:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.002+0000 7fa353400640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 0x7fa3380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.002+0000 7fa353400640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 0x7fa3380a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa354104970 tx=0x7fa33c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.003+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa344022070 con 0x7fa354100a80 2024-04-26T22:00:11.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:00:10.754+0000 7f6d33a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:00:11.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:00:10.754+0000 7f6d33a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:00:11.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:00:10.754+0000 7f6d33a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:00:11.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:00:10.754+0000 7f6d33a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:00:11.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:00:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:00:10.754+0000 7f6d33a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:00:11.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.243+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa3541089c0 con 0x7fa3380a3bb0 2024-04-26T22:00:11.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.286+0000 7fa350c00640 1 -- 172.21.15.12:0/1693505811 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa3541089c0 con 0x7fa3380a3bb0 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 msgr2=0x7fa3380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 0x7fa3380a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa354104970 tx=0x7fa33c005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 msgr2=0x7fa354104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 0x7fa354104c00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa3440097d0 tx=0x7fa344009c90 comp rx=0 tx=0).stop 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 shutdown_connections 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa3380a3bb0 0x7fa3380a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa354102630 0x7fa354103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa354101c70 0x7fa354103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 --2- 172.21.15.12:0/1693505811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa354100a80 0x7fa354104c00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 >> 172.21.15.12:0/1693505811 conn(0x7fa3540fbff0 msgr2=0x7fa3541072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 shutdown_connections 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.289+0000 7fa359c00640 1 -- 172.21.15.12:0/1693505811 wait complete. 2024-04-26T22:00:11.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.533+0000 7f415ae00640 1 -- 172.21.15.12:0/2541822201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f415410a780 msgr2=0x7f415410cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.533+0000 7f415ae00640 1 --2- 172.21.15.12:0/2541822201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f415410a780 0x7f415410cc10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f414c009be0 tx=0x7f414c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 -- 172.21.15.12:0/2541822201 shutdown_connections 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 --2- 172.21.15.12:0/2541822201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f415410a780 0x7f415410cc10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 --2- 172.21.15.12:0/2541822201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f4154075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 --2- 172.21.15.12:0/2541822201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4154076970 0x7f4154074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 -- 172.21.15.12:0/2541822201 >> 172.21.15.12:0/2541822201 conn(0x7f41540fde30 msgr2=0x7f4154100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 -- 172.21.15.12:0/2541822201 shutdown_connections 2024-04-26T22:00:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.534+0000 7f415ae00640 1 -- 172.21.15.12:0/2541822201 wait complete. 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 Processor -- start 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 -- start start 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f415419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4154076970 0x7f415419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4154110540 con 0x7f4154075390 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41541103c0 con 0x7f4154076970 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.535+0000 7f415ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41541106c0 con 0x7f415410a780 2024-04-26T22:00:11.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f4159a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f415419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43472/0 (socket says 172.21.15.12:43472) 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 -- 172.21.15.12:0/3897712199 learned_addr learned my addr 172.21.15.12:0/3897712199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f4159000640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4154076970 0x7f415419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 -- 172.21.15.12:0/3897712199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4154076970 msgr2=0x7f415419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4154076970 0x7f415419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 -- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 msgr2=0x7f415419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f415419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 -- 172.21.15.12:0/3897712199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541a2840 con 0x7f415410a780 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f4159a00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f415419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:11.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.536+0000 7f415a400640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f414c034040 tx=0x7f414c030d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.537+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c041020 con 0x7f415410a780 2024-04-26T22:00:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.537+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541a2ad0 con 0x7f415410a780 2024-04-26T22:00:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.537+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414c0318f0 con 0x7f415410a780 2024-04-26T22:00:11.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.537+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c036040 con 0x7f415410a780 2024-04-26T22:00:11.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.537+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541a3010 con 0x7f415410a780 2024-04-26T22:00:11.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.538+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41541094b0 con 0x7f415410a780 2024-04-26T22:00:11.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.540+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f414c044020 con 0x7f415410a780 2024-04-26T22:00:11.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.542+0000 7f4152000640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 0x7f41380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.542+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f414c0f1490 con 0x7f415410a780 2024-04-26T22:00:11.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.542+0000 7f4159a00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 0x7f41380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.543+0000 7f4159a00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 0x7f41380a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f415419c100 tx=0x7f4144009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.546+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f414c04b020 con 0x7f415410a780 2024-04-26T22:00:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:11 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:00:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:11 smithi012 ceph-mon[28184]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:11 smithi012 ceph-mon[28184]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:11 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:00:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:11 smithi154 ceph-mon[32578]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:11.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:11 smithi154 ceph-mon[32578]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:11 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T22:00:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:11 smithi077 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T22:00:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:11 smithi077 ceph-mon[32280]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:11.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.786+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4154062e00 con 0x7f41380a3bb0 2024-04-26T22:00:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.794+0000 7f4152000640 1 -- 172.21.15.12:0/3897712199 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4154062e00 con 0x7f41380a3bb0 2024-04-26T22:00:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/ed40312c-9c59-4382-9aac-9a5e4589a29b 2024-04-26T22:00:11.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 msgr2=0x7f41380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 0x7f41380a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f415419c100 tx=0x7f4144009260 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 msgr2=0x7f41541a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f414c034040 tx=0x7f414c030d70 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 shutdown_connections 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41380a3bb0 0x7f41380a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f415410a780 0x7f41541a21a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4154076970 0x7f415419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 --2- 172.21.15.12:0/3897712199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4154075390 0x7f415419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.796+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 >> 172.21.15.12:0/3897712199 conn(0x7f41540fde30 msgr2=0x7f4154106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.797+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 shutdown_connections 2024-04-26T22:00:11.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:11.797+0000 7f415ae00640 1 -- 172.21.15.12:0/3897712199 wait complete. 2024-04-26T22:00:11.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 -- 172.21.15.12:0/1544817307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 msgr2=0x7f6030109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 --2- 172.21.15.12:0/1544817307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f6030109290 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f602002a400 comp rx=0 tx=0).stop 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 -- 172.21.15.12:0/1544817307 shutdown_connections 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 --2- 172.21.15.12:0/1544817307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f603010a980 0x7f60301112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 --2- 172.21.15.12:0/1544817307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6030109fc0 0x7f603010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 --2- 172.21.15.12:0/1544817307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f6030109290 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 -- 172.21.15.12:0/1544817307 >> 172.21.15.12:0/1544817307 conn(0x7f6030077e00 msgr2=0x7f603007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 -- 172.21.15.12:0/1544817307 shutdown_connections 2024-04-26T22:00:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.044+0000 7f602f200640 1 -- 172.21.15.12:0/1544817307 wait complete. 2024-04-26T22:00:12.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 Processor -- start 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 -- start start 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f603019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 0x7f603019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f603010a980 0x7f60301a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6030114a10 con 0x7f6030108e70 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6030114890 con 0x7f6030109fc0 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.045+0000 7f602f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6030114b90 con 0x7f603010a980 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f602de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f603019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f602de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f603019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43592/0 (socket says 172.21.15.12:43592) 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 0x7f603019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f602de00640 1 -- 172.21.15.12:0/3123658044 learned_addr learned my addr 172.21.15.12:0/3123658044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f602e800640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f603010a980 0x7f60301a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 -- 172.21.15.12:0/3123658044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f603010a980 msgr2=0x7f60301a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f603010a980 0x7f60301a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 -- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 msgr2=0x7f603019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f603019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.046+0000 7f6027e00640 1 -- 172.21.15.12:0/3123658044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301a6cd0 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f6027e00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 0x7f603019f5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f601800e9c0 tx=0x7f601800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601800ccc0 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60180035f0 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301a6fc0 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601800ce60 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.047+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a7480 con 0x7f6030109fc0 2024-04-26T22:00:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.048+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030110180 con 0x7f6030109fc0 2024-04-26T22:00:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.050+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6018004000 con 0x7f6030109fc0 2024-04-26T22:00:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.052+0000 7f6026a00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 0x7f60140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.052+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f60180d0960 con 0x7f6030109fc0 2024-04-26T22:00:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.052+0000 7f602de00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 0x7f60140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.053+0000 7f602de00640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 0x7f60140a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60200096c0 tx=0x7f6020005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.056+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f601809d0a0 con 0x7f6030109fc0 2024-04-26T22:00:12.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.277+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6018099010 con 0x7f6030109fc0 2024-04-26T22:00:12.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.312+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6030062d70 con 0x7f60140a3c30 2024-04-26T22:00:12.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.357+0000 7f6026a00640 1 -- 172.21.15.12:0/3123658044 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6030062d70 con 0x7f60140a3c30 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.360+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 msgr2=0x7f60140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.360+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 0x7f60140a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60200096c0 tx=0x7f6020005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 msgr2=0x7f603019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 0x7f603019f5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f601800e9c0 tx=0x7f601800ee80 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 shutdown_connections 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f60140a3c30 0x7f60140a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f603010a980 0x7f60301a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030109fc0 0x7f603019f5f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 --2- 172.21.15.12:0/3123658044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030108e70 0x7f603019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 >> 172.21.15.12:0/3123658044 conn(0x7f6030077e00 msgr2=0x7f603010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 shutdown_connections 2024-04-26T22:00:12.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.361+0000 7f602f200640 1 -- 172.21.15.12:0/3123658044 wait complete. 2024-04-26T22:00:12.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T22:00:12.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:12 smithi012 ceph-mon[28184]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 426 B/s wr, 0 op/s 2024-04-26T22:00:12.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:12 smithi012 ceph-mon[28184]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:12 smithi012 ceph-mon[28184]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.605+0000 7f22f3200640 1 -- 172.21.15.12:0/3356751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 msgr2=0x7f22f4102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.605+0000 7f22f3200640 1 --2- 172.21.15.12:0/3356751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f4102890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009860 tx=0x7f22ec02a340 comp rx=0 tx=0).stop 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 -- 172.21.15.12:0/3356751876 shutdown_connections 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 --2- 172.21.15.12:0/3356751876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f4102890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 --2- 172.21.15.12:0/3356751876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101a70 0x7f22f4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 --2- 172.21.15.12:0/3356751876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101110 0x7f22f4101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 -- 172.21.15.12:0/3356751876 >> 172.21.15.12:0/3356751876 conn(0x7f22f40fccd0 msgr2=0x7f22f40ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 -- 172.21.15.12:0/3356751876 shutdown_connections 2024-04-26T22:00:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.606+0000 7f22f3200640 1 -- 172.21.15.12:0/3356751876 wait complete. 2024-04-26T22:00:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 Processor -- start 2024-04-26T22:00:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 -- start start 2024-04-26T22:00:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 0x7f22f419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f41a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4110540 con 0x7f22f4102410 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41103c0 con 0x7f22f4101110 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f3200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41106c0 con 0x7f22f4101a70 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.607+0000 7f22f2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f41a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 0x7f22f419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f41a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43614/0 (socket says 172.21.15.12:43614) 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43506/0 (socket says 172.21.15.12:43506) 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 -- 172.21.15.12:0/3230942768 learned_addr learned my addr 172.21.15.12:0/3230942768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 0x7f22f419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58092/0 (socket says 172.21.15.12:58092) 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 -- 172.21.15.12:0/3230942768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 msgr2=0x7f22f419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 0x7f22f419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 -- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 msgr2=0x7f22f41a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f41a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 -- 172.21.15.12:0/3230942768 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f41a2810 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22f1400640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009d50 tx=0x7f22dc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.608+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003b70 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.609+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22dc003d10 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.609+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0108a0 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.609+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a2b00 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.609+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a3000 con 0x7f22f4101a70 2024-04-26T22:00:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.610+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f410c040 con 0x7f22f4101a70 2024-04-26T22:00:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.612+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22dc004000 con 0x7f22f4101a70 2024-04-26T22:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.613+0000 7f22eaa00640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 0x7f22d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.614+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f22dc0d0fe0 con 0x7f22f4101a70 2024-04-26T22:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.614+0000 7f22f1e00640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 0x7f22d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.614+0000 7f22f1e00640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 0x7f22d40a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009530 tx=0x7f22e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.616+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22dc09d5e0 con 0x7f22f4101a70 2024-04-26T22:00:12.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:12 smithi154 ceph-mon[32578]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 426 B/s wr, 0 op/s 2024-04-26T22:00:12.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:12 smithi154 ceph-mon[32578]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:12 smithi154 ceph-mon[32578]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:12 smithi077 ceph-mon[32280]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 426 B/s wr, 0 op/s 2024-04-26T22:00:12.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:12 smithi077 ceph-mon[32280]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:12 smithi077 ceph-mon[32280]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:12.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.855+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f22f4062d70 con 0x7f22d40a3cc0 2024-04-26T22:00:12.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.865+0000 7f22eaa00640 1 -- 172.21.15.12:0/3230942768 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f22f4062d70 con 0x7f22d40a3cc0 2024-04-26T22:00:12.866 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/6437fa6c-391f-4fc8-92f8-759348360930 2024-04-26T22:00:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 msgr2=0x7f22d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 0x7f22d40a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009530 tx=0x7f22e4009260 comp rx=0 tx=0).stop 2024-04-26T22:00:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 msgr2=0x7f22f419b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009d50 tx=0x7f22dc00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 shutdown_connections 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f22d40a3cc0 0x7f22d40a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f4102410 0x7f22f41a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101a70 0x7f22f419b130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 --2- 172.21.15.12:0/3230942768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f4101110 0x7f22f419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 >> 172.21.15.12:0/3230942768 conn(0x7f22f40fccd0 msgr2=0x7f22f40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 shutdown_connections 2024-04-26T22:00:12.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:12.868+0000 7f22f3200640 1 -- 172.21.15.12:0/3230942768 wait complete. 2024-04-26T22:00:12.889 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 -- 172.21.15.12:0/2853178034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44108d90 msgr2=0x7f5b441091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 --2- 172.21.15.12:0/2853178034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44108d90 0x7f5b441091b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b3802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 -- 172.21.15.12:0/2853178034 shutdown_connections 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 --2- 172.21.15.12:0/2853178034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b4410a940 0x7f5b441111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 --2- 172.21.15.12:0/2853178034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b44109f80 0x7f5b4410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 --2- 172.21.15.12:0/2853178034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44108d90 0x7f5b441091b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 -- 172.21.15.12:0/2853178034 >> 172.21.15.12:0/2853178034 conn(0x7f5b44077d30 msgr2=0x7f5b4407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 -- 172.21.15.12:0/2853178034 shutdown_connections 2024-04-26T22:00:13.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.106+0000 7f5b4b200640 1 -- 172.21.15.12:0/2853178034 wait complete. 2024-04-26T22:00:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.107+0000 7f5b4b200640 1 Processor -- start 2024-04-26T22:00:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.107+0000 7f5b4b200640 1 -- start start 2024-04-26T22:00:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b44108d90 0x7f5b4419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4410a940 0x7f5b441a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441148c0 con 0x7f5b44109f80 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44114740 con 0x7f5b4410a940 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44114a40 con 0x7f5b44108d90 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b4a800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4410a940 0x7f5b441a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43636/0 (socket says 172.21.15.12:43636) 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 -- 172.21.15.12:0/845132530 learned_addr learned my addr 172.21.15.12:0/845132530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49e00640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b44108d90 0x7f5b4419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 -- 172.21.15.12:0/845132530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b44108d90 msgr2=0x7f5b4419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b44108d90 0x7f5b4419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 -- 172.21.15.12:0/845132530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4410a940 msgr2=0x7f5b441a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4410a940 0x7f5b441a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.108+0000 7f5b49400640 1 -- 172.21.15.12:0/845132530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b441a6bb0 con 0x7f5b44109f80 2024-04-26T22:00:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b49400640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400ea60 tx=0x7f5b3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cde0 con 0x7f5b44109f80 2024-04-26T22:00:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b340035f0 con 0x7f5b44109f80 2024-04-26T22:00:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34010690 con 0x7f5b44109f80 2024-04-26T22:00:13.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b441a6ea0 con 0x7f5b44109f80 2024-04-26T22:00:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.109+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441a7360 con 0x7f5b44109f80 2024-04-26T22:00:13.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.110+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44110180 con 0x7f5b44109f80 2024-04-26T22:00:13.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.113+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5b340037a0 con 0x7f5b44109f80 2024-04-26T22:00:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.114+0000 7f5b3ea00640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 0x7f5b2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.115+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5b340d0ad0 con 0x7f5b44109f80 2024-04-26T22:00:13.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.115+0000 7f5b49e00640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 0x7f5b2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.116+0000 7f5b49e00640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 0x7f5b2c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009230 tx=0x7f5b380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.120+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b3409d0b0 con 0x7f5b44109f80 2024-04-26T22:00:13.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.373+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b4402dad0 con 0x7f5b44109f80 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.374+0000 7f5b3ea00640 1 -- 172.21.15.12:0/845132530 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7f5b3409c9b0 con 0x7f5b44109f80 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.376+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 msgr2=0x7f5b2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.376+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 0x7f5b2c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009230 tx=0x7f5b380314a0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.376+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 msgr2=0x7f5b4419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.376+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400ea60 tx=0x7f5b3400ef20 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 shutdown_connections 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5b2c0a3c40 0x7f5b2c0a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4410a940 0x7f5b441a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b44109f80 0x7f5b4419f4d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 --2- 172.21.15.12:0/845132530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b44108d90 0x7f5b4419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 >> 172.21.15.12:0/845132530 conn(0x7f5b44077d30 msgr2=0x7f5b4410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 shutdown_connections 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.377+0000 7f5b4b200640 1 -- 172.21.15.12:0/845132530 wait complete. 2024-04-26T22:00:13.397 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:13 smithi012 ceph-mon[28184]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:00:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:13 smithi012 ceph-mon[28184]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:13 smithi012 ceph-mon[28184]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 -- 172.21.15.12:0/1406258020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850104990 msgr2=0x7fe850104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 --2- 172.21.15.12:0/1406258020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850104990 0x7fe850104db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe844009900 tx=0x7fe84402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 -- 172.21.15.12:0/1406258020 shutdown_connections 2024-04-26T22:00:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 --2- 172.21.15.12:0/1406258020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850106540 0x7fe85010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 --2- 172.21.15.12:0/1406258020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 0x7fe850106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 --2- 172.21.15.12:0/1406258020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850104990 0x7fe850104db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 -- 172.21.15.12:0/1406258020 >> 172.21.15.12:0/1406258020 conn(0x7fe8500fff00 msgr2=0x7fe850102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 -- 172.21.15.12:0/1406258020 shutdown_connections 2024-04-26T22:00:13.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.614+0000 7fe856200640 1 -- 172.21.15.12:0/1406258020 wait complete. 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 Processor -- start 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 -- start start 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850104990 0x7fe85019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 0x7fe85019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.615+0000 7fe856200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe850110490 con 0x7fe850106540 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe856200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe850110310 con 0x7fe850104990 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe856200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe850110610 con 0x7fe850105b80 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe854e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850104990 0x7fe85019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43652/0 (socket says 172.21.15.12:43652) 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 -- 172.21.15.12:0/1579251453 learned_addr learned my addr 172.21.15.12:0/1579251453 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe84fe00640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 0x7fe85019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 -- 172.21.15.12:0/1579251453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 msgr2=0x7fe85019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 0x7fe85019b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 -- 172.21.15.12:0/1579251453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850104990 msgr2=0x7fe85019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850104990 0x7fe85019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.616+0000 7fe855800640 1 -- 172.21.15.12:0/1579251453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8501a27b0 con 0x7fe850106540 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe855800640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe84800eb10 tx=0x7fe84800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848004b20 con 0x7fe850106540 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe848004cc0 con 0x7fe850106540 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848010670 con 0x7fe850106540 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8501a2aa0 con 0x7fe850106540 2024-04-26T22:00:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.617+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8501a2fa0 con 0x7fe850106540 2024-04-26T22:00:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.618+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85010bdd0 con 0x7fe850106540 2024-04-26T22:00:13.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.620+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe848005020 con 0x7fe850106540 2024-04-26T22:00:13.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.622+0000 7fe84d600640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 0x7fe8380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.622+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe8480d0d60 con 0x7fe850106540 2024-04-26T22:00:13.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.622+0000 7fe854e00640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 0x7fe8380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.623+0000 7fe854e00640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 0x7fe8380a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe844009230 tx=0x7fe8440314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.626+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe84809d340 con 0x7fe850106540 2024-04-26T22:00:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:13 smithi154 ceph-mon[32578]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:00:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:13 smithi154 ceph-mon[32578]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:13 smithi154 ceph-mon[32578]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:13 smithi077 ceph-mon[32280]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T22:00:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:13 smithi077 ceph-mon[32280]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:13 smithi077 ceph-mon[32280]: from='client.24853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:00:13.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.837+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe85002dcf0 con 0x7fe850106540 2024-04-26T22:00:13.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.842+0000 7fe84d600640 1 -- 172.21.15.12:0/1579251453 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7fe84809cc40 con 0x7fe850106540 2024-04-26T22:00:13.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:13.844 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 msgr2=0x7fe8380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 0x7fe8380a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe844009230 tx=0x7fe8440314a0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 msgr2=0x7fe8501a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe84800eb10 tx=0x7fe84800c2d0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 shutdown_connections 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe8380a3cc0 0x7fe8380a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe850106540 0x7fe8501a2140 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe850105b80 0x7fe85019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 --2- 172.21.15.12:0/1579251453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe850104990 0x7fe85019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 >> 172.21.15.12:0/1579251453 conn(0x7fe8500fff00 msgr2=0x7fe850108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 shutdown_connections 2024-04-26T22:00:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:13.845+0000 7fe856200640 1 -- 172.21.15.12:0/1579251453 wait complete. 2024-04-26T22:00:13.866 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:00:13.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 -- 172.21.15.12:0/806848046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c108d90 msgr2=0x7fd48c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 --2- 172.21.15.12:0/806848046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c108d90 0x7fd48c1091b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009900 tx=0x7fd47c02a410 comp rx=0 tx=0).stop 2024-04-26T22:00:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 -- 172.21.15.12:0/806848046 shutdown_connections 2024-04-26T22:00:14.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 --2- 172.21.15.12:0/806848046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c10a940 0x7fd48c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 --2- 172.21.15.12:0/806848046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c109f80 0x7fd48c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 --2- 172.21.15.12:0/806848046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c108d90 0x7fd48c1091b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 -- 172.21.15.12:0/806848046 >> 172.21.15.12:0/806848046 conn(0x7fd48c077d30 msgr2=0x7fd48c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 -- 172.21.15.12:0/806848046 shutdown_connections 2024-04-26T22:00:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.082+0000 7fd491600640 1 -- 172.21.15.12:0/806848046 wait complete. 2024-04-26T22:00:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.083+0000 7fd491600640 1 Processor -- start 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 -- start start 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 0x7fd48c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10a940 0x7fd48c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1148e0 con 0x7fd48c10a940 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c114760 con 0x7fd48c1a4710 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd491600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c114a60 con 0x7fd48c109f80 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd48b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10a940 0x7fd48c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd48be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 0x7fd48c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd490c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd48be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 0x7fd48c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43566/0 (socket says 172.21.15.12:43566) 2024-04-26T22:00:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd490c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:58144/0 (socket says 172.21.15.12:58144) 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.084+0000 7fd48be00640 1 -- 172.21.15.12:0/438537144 learned_addr learned my addr 172.21.15.12:0/438537144 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 -- 172.21.15.12:0/438537144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 msgr2=0x7fd48c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 0x7fd48c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 -- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10a940 msgr2=0x7fd48c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10a940 0x7fd48c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 -- 172.21.15.12:0/438537144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a7170 con 0x7fd48c1a4710 2024-04-26T22:00:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd490c00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400ce20 con 0x7fd48c1a4710 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484005020 con 0x7fd48c1a4710 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484010690 con 0x7fd48c1a4710 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.085+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a7460 con 0x7fd48c1a4710 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.086+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a7960 con 0x7fd48c1a4710 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.087+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c110270 con 0x7fd48c1a4710 2024-04-26T22:00:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.089+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd4840054d0 con 0x7fd48c1a4710 2024-04-26T22:00:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.091+0000 7fd488c00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 0x7fd4700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.091+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd4840d12a0 con 0x7fd48c1a4710 2024-04-26T22:00:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.091+0000 7fd48be00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 0x7fd4700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.092+0000 7fd48be00640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 0x7fd4700a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd47c031500 tx=0x7fd47c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.092+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd48409d8e0 con 0x7fd48c1a4710 2024-04-26T22:00:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.293+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd48c062d70 con 0x7fd4700a3cc0 2024-04-26T22:00:14.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.330+0000 7fd488c00640 1 -- 172.21.15.12:0/438537144 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340218 (secure 0 0 0) 0x7fd48c062d70 con 0x7fd4700a3cc0 2024-04-26T22:00:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:14.339 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.338+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 msgr2=0x7fd4700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 0x7fd4700a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd47c031500 tx=0x7fd47c005a20 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 msgr2=0x7fd48c1a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400ef10 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 shutdown_connections 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cc0 0x7fd4700a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c1a4710 0x7fd48c1a6b00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10a940 0x7fd48c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 --2- 172.21.15.12:0/438537144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c109f80 0x7fd48c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 >> 172.21.15.12:0/438537144 conn(0x7fd48c077d30 msgr2=0x7fd48c10d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 shutdown_connections 2024-04-26T22:00:14.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.339+0000 7fd491600640 1 -- 172.21.15.12:0/438537144 wait complete. 2024-04-26T22:00:14.364 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-04-26T22:00:13.130613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595125,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":145,"num_read_kb":130,"num_write":235,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379344,"kb_used_data":13524,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562480,"statfs":{"total":1151940427776,"available":1151551979520,"internally_reserved":0,"allocated":13848576,"data_stored":7590108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1898,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":11,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005698"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149711+0000","last_change":"2024-04-26T21:59:44.048850+0000","last_active":"2024-04-26T21:59:51.149711+0000","last_peered":"2024-04-26T21:59:51.149711+0000","last_clean":"2024-04-26T21:59:51.149711+0000","last_became_active":"2024-04-26T21:59:44.048521+0000","last_became_peered":"2024-04-26T21:59:44.048521+0000","last_unstale":"2024-04-26T21:59:51.149711+0000","last_undegraded":"2024-04-26T21:59:51.149711+0000","last_fullsized":"2024-04-26T21:59:51.149711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:09.031610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324469+0000","last_change":"2024-04-26T21:59:44.048900+0000","last_active":"2024-04-26T21:59:50.324469+0000","last_peered":"2024-04-26T21:59:50.324469+0000","last_clean":"2024-04-26T21:59:50.324469+0000","last_became_active":"2024-04-26T21:59:44.048475+0000","last_became_peered":"2024-04-26T21:59:44.048475+0000","last_unstale":"2024-04-26T21:59:50.324469+0000","last_undegraded":"2024-04-26T21:59:50.324469+0000","last_fullsized":"2024-04-26T21:59:50.324469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:01.724256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324301+0000","last_change":"2024-04-26T21:59:44.045875+0000","last_active":"2024-04-26T21:59:50.324301+0000","last_peered":"2024-04-26T21:59:50.324301+0000","last_clean":"2024-04-26T21:59:50.324301+0000","last_became_active":"2024-04-26T21:59:44.045413+0000","last_became_peered":"2024-04-26T21:59:44.045413+0000","last_unstale":"2024-04-26T21:59:50.324301+0000","last_undegraded":"2024-04-26T21:59:50.324301+0000","last_fullsized":"2024-04-26T21:59:50.324301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:19.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118365+0000","last_change":"2024-04-26T21:59:44.042513+0000","last_active":"2024-04-26T21:59:49.118365+0000","last_peered":"2024-04-26T21:59:49.118365+0000","last_clean":"2024-04-26T21:59:49.118365+0000","last_became_active":"2024-04-26T21:59:44.042156+0000","last_became_peered":"2024-04-26T21:59:44.042156+0000","last_unstale":"2024-04-26T21:59:49.118365+0000","last_undegraded":"2024-04-26T21:59:49.118365+0000","last_fullsized":"2024-04-26T21:59:49.118365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:08.494602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719554+0000","last_change":"2024-04-26T21:59:44.063199+0000","last_active":"2024-04-26T21:59:50.719554+0000","last_peered":"2024-04-26T21:59:50.719554+0000","last_clean":"2024-04-26T21:59:50.719554+0000","last_became_active":"2024-04-26T21:59:44.062845+0000","last_became_peered":"2024-04-26T21:59:44.062845+0000","last_unstale":"2024-04-26T21:59:50.719554+0000","last_undegraded":"2024-04-26T21:59:50.719554+0000","last_fullsized":"2024-04-26T21:59:50.719554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:36.247652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388977+0000","last_change":"2024-04-26T21:59:44.058069+0000","last_active":"2024-04-26T21:59:50.388977+0000","last_peered":"2024-04-26T21:59:50.388977+0000","last_clean":"2024-04-26T21:59:50.388977+0000","last_became_active":"2024-04-26T21:59:44.057726+0000","last_became_peered":"2024-04-26T21:59:44.057726+0000","last_unstale":"2024-04-26T21:59:50.388977+0000","last_undegraded":"2024-04-26T21:59:50.388977+0000","last_fullsized":"2024-04-26T21:59:50.388977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:37.887034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661192+0000","last_change":"2024-04-26T21:59:44.046887+0000","last_active":"2024-04-26T21:59:50.661192+0000","last_peered":"2024-04-26T21:59:50.661192+0000","last_clean":"2024-04-26T21:59:50.661192+0000","last_became_active":"2024-04-26T21:59:44.046507+0000","last_became_peered":"2024-04-26T21:59:44.046507+0000","last_unstale":"2024-04-26T21:59:50.661192+0000","last_undegraded":"2024-04-26T21:59:50.661192+0000","last_fullsized":"2024-04-26T21:59:50.661192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:03.312480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119800+0000","last_change":"2024-04-26T21:59:44.060437+0000","last_active":"2024-04-26T21:59:49.119800+0000","last_peered":"2024-04-26T21:59:49.119800+0000","last_clean":"2024-04-26T21:59:49.119800+0000","last_became_active":"2024-04-26T21:59:44.055341+0000","last_became_peered":"2024-04-26T21:59:44.055341+0000","last_unstale":"2024-04-26T21:59:49.119800+0000","last_undegraded":"2024-04-26T21:59:49.119800+0000","last_fullsized":"2024-04-26T21:59:49.119800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:22.452619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720226+0000","last_change":"2024-04-26T21:59:44.043537+0000","last_active":"2024-04-26T21:59:50.720226+0000","last_peered":"2024-04-26T21:59:50.720226+0000","last_clean":"2024-04-26T21:59:50.720226+0000","last_became_active":"2024-04-26T21:59:44.042227+0000","last_became_peered":"2024-04-26T21:59:44.042227+0000","last_unstale":"2024-04-26T21:59:50.720226+0000","last_undegraded":"2024-04-26T21:59:50.720226+0000","last_fullsized":"2024-04-26T21:59:50.720226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.663380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389294+0000","last_change":"2024-04-26T21:59:44.039338+0000","last_active":"2024-04-26T21:59:50.389294+0000","last_peered":"2024-04-26T21:59:50.389294+0000","last_clean":"2024-04-26T21:59:50.389294+0000","last_became_active":"2024-04-26T21:59:44.039000+0000","last_became_peered":"2024-04-26T21:59:44.039000+0000","last_unstale":"2024-04-26T21:59:50.389294+0000","last_undegraded":"2024-04-26T21:59:50.389294+0000","last_fullsized":"2024-04-26T21:59:50.389294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:20.829507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661671+0000","last_change":"2024-04-26T21:59:44.046272+0000","last_active":"2024-04-26T21:59:50.661671+0000","last_peered":"2024-04-26T21:59:50.661671+0000","last_clean":"2024-04-26T21:59:50.661671+0000","last_became_active":"2024-04-26T21:59:44.045235+0000","last_became_peered":"2024-04-26T21:59:44.045235+0000","last_unstale":"2024-04-26T21:59:50.661671+0000","last_undegraded":"2024-04-26T21:59:50.661671+0000","last_fullsized":"2024-04-26T21:59:50.661671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118216+0000","last_change":"2024-04-26T21:59:44.044698+0000","last_active":"2024-04-26T21:59:49.118216+0000","last_peered":"2024-04-26T21:59:49.118216+0000","last_clean":"2024-04-26T21:59:49.118216+0000","last_became_active":"2024-04-26T21:59:44.043394+0000","last_became_peered":"2024-04-26T21:59:44.043394+0000","last_unstale":"2024-04-26T21:59:49.118216+0000","last_undegraded":"2024-04-26T21:59:49.118216+0000","last_fullsized":"2024-04-26T21:59:49.118216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:55.254581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723599+0000","last_change":"2024-04-26T21:59:44.060896+0000","last_active":"2024-04-26T21:59:50.723599+0000","last_peered":"2024-04-26T21:59:50.723599+0000","last_clean":"2024-04-26T21:59:50.723599+0000","last_became_active":"2024-04-26T21:59:44.060010+0000","last_became_peered":"2024-04-26T21:59:44.060010+0000","last_unstale":"2024-04-26T21:59:50.723599+0000","last_undegraded":"2024-04-26T21:59:50.723599+0000","last_fullsized":"2024-04-26T21:59:50.723599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:46.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323909+0000","last_change":"2024-04-26T21:59:44.050761+0000","last_active":"2024-04-26T21:59:50.323909+0000","last_peered":"2024-04-26T21:59:50.323909+0000","last_clean":"2024-04-26T21:59:50.323909+0000","last_became_active":"2024-04-26T21:59:44.050229+0000","last_became_peered":"2024-04-26T21:59:44.050229+0000","last_unstale":"2024-04-26T21:59:50.323909+0000","last_undegraded":"2024-04-26T21:59:50.323909+0000","last_fullsized":"2024-04-26T21:59:50.323909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:46.114433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323698+0000","last_change":"2024-04-26T21:59:44.056593+0000","last_active":"2024-04-26T21:59:50.323698+0000","last_peered":"2024-04-26T21:59:50.323698+0000","last_clean":"2024-04-26T21:59:50.323698+0000","last_became_active":"2024-04-26T21:59:44.056111+0000","last_became_peered":"2024-04-26T21:59:44.056111+0000","last_unstale":"2024-04-26T21:59:50.323698+0000","last_undegraded":"2024-04-26T21:59:50.323698+0000","last_fullsized":"2024-04-26T21:59:50.323698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:34.118566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719667+0000","last_change":"2024-04-26T21:59:44.049616+0000","last_active":"2024-04-26T21:59:50.719667+0000","last_peered":"2024-04-26T21:59:50.719667+0000","last_clean":"2024-04-26T21:59:50.719667+0000","last_became_active":"2024-04-26T21:59:44.049259+0000","last_became_peered":"2024-04-26T21:59:44.049259+0000","last_unstale":"2024-04-26T21:59:50.719667+0000","last_undegraded":"2024-04-26T21:59:50.719667+0000","last_fullsized":"2024-04-26T21:59:50.719667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:25.264219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149586+0000","last_change":"2024-04-26T21:59:44.048657+0000","last_active":"2024-04-26T21:59:51.149586+0000","last_peered":"2024-04-26T21:59:51.149586+0000","last_clean":"2024-04-26T21:59:51.149586+0000","last_became_active":"2024-04-26T21:59:44.048209+0000","last_became_peered":"2024-04-26T21:59:44.048209+0000","last_unstale":"2024-04-26T21:59:51.149586+0000","last_undegraded":"2024-04-26T21:59:51.149586+0000","last_fullsized":"2024-04-26T21:59:51.149586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:01.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459579+0000","last_change":"2024-04-26T21:59:44.058215+0000","last_active":"2024-04-26T21:59:49.459579+0000","last_peered":"2024-04-26T21:59:49.459579+0000","last_clean":"2024-04-26T21:59:49.459579+0000","last_became_active":"2024-04-26T21:59:44.057422+0000","last_became_peered":"2024-04-26T21:59:44.057422+0000","last_unstale":"2024-04-26T21:59:49.459579+0000","last_undegraded":"2024-04-26T21:59:49.459579+0000","last_fullsized":"2024-04-26T21:59:49.459579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:17.215168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313975+0000","last_change":"2024-04-26T21:59:44.054840+0000","last_active":"2024-04-26T21:59:50.313975+0000","last_peered":"2024-04-26T21:59:50.313975+0000","last_clean":"2024-04-26T21:59:50.313975+0000","last_became_active":"2024-04-26T21:59:44.054629+0000","last_became_peered":"2024-04-26T21:59:44.054629+0000","last_unstale":"2024-04-26T21:59:50.313975+0000","last_undegraded":"2024-04-26T21:59:50.313975+0000","last_fullsized":"2024-04-26T21:59:50.313975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:37.204066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313510+0000","last_change":"2024-04-26T21:59:48.145585+0000","last_active":"2024-04-26T21:59:50.313510+0000","last_peered":"2024-04-26T21:59:50.313510+0000","last_clean":"2024-04-26T21:59:50.313510+0000","last_became_active":"2024-04-26T21:59:48.145283+0000","last_became_peered":"2024-04-26T21:59:48.145283+0000","last_unstale":"2024-04-26T21:59:50.313510+0000","last_undegraded":"2024-04-26T21:59:50.313510+0000","last_fullsized":"2024-04-26T21:59:50.313510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:53.096333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.460095+0000","last_change":"2024-04-26T21:59:43.059127+0000","last_active":"2024-04-26T21:59:49.460095+0000","last_peered":"2024-04-26T21:59:49.460095+0000","last_clean":"2024-04-26T21:59:49.460095+0000","last_became_active":"2024-04-26T21:59:43.057996+0000","last_became_peered":"2024-04-26T21:59:43.057996+0000","last_unstale":"2024-04-26T21:59:49.460095+0000","last_undegraded":"2024-04-26T21:59:49.460095+0000","last_fullsized":"2024-04-26T21:59:49.460095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:06.086437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121706+0000","last_change":"2024-04-26T21:59:44.060192+0000","last_active":"2024-04-26T21:59:49.121706+0000","last_peered":"2024-04-26T21:59:49.121706+0000","last_clean":"2024-04-26T21:59:49.121706+0000","last_became_active":"2024-04-26T21:59:44.054969+0000","last_became_peered":"2024-04-26T21:59:44.054969+0000","last_unstale":"2024-04-26T21:59:49.121706+0000","last_undegraded":"2024-04-26T21:59:49.121706+0000","last_fullsized":"2024-04-26T21:59:49.121706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:54.670629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117835+0000","last_change":"2024-04-26T21:59:48.144947+0000","last_active":"2024-04-26T21:59:49.117835+0000","last_peered":"2024-04-26T21:59:49.117835+0000","last_clean":"2024-04-26T21:59:49.117835+0000","last_became_active":"2024-04-26T21:59:48.144547+0000","last_became_peered":"2024-04-26T21:59:48.144547+0000","last_unstale":"2024-04-26T21:59:49.117835+0000","last_undegraded":"2024-04-26T21:59:49.117835+0000","last_fullsized":"2024-04-26T21:59:49.117835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.492136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122109+0000","last_change":"2024-04-26T21:59:43.071253+0000","last_active":"2024-04-26T21:59:49.122109+0000","last_peered":"2024-04-26T21:59:49.122109+0000","last_clean":"2024-04-26T21:59:49.122109+0000","last_became_active":"2024-04-26T21:59:43.070926+0000","last_became_peered":"2024-04-26T21:59:43.070926+0000","last_unstale":"2024-04-26T21:59:49.122109+0000","last_undegraded":"2024-04-26T21:59:49.122109+0000","last_fullsized":"2024-04-26T21:59:49.122109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:13.280936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324550+0000","last_change":"2024-04-26T21:59:44.051951+0000","last_active":"2024-04-26T21:59:50.324550+0000","last_peered":"2024-04-26T21:59:50.324550+0000","last_clean":"2024-04-26T21:59:50.324550+0000","last_became_active":"2024-04-26T21:59:44.051206+0000","last_became_peered":"2024-04-26T21:59:44.051206+0000","last_unstale":"2024-04-26T21:59:50.324550+0000","last_undegraded":"2024-04-26T21:59:50.324550+0000","last_fullsized":"2024-04-26T21:59:50.324550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:05.034106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117969+0000","last_change":"2024-04-26T21:59:48.150369+0000","last_active":"2024-04-26T21:59:49.117969+0000","last_peered":"2024-04-26T21:59:49.117969+0000","last_clean":"2024-04-26T21:59:49.117969+0000","last_became_active":"2024-04-26T21:59:48.149632+0000","last_became_peered":"2024-04-26T21:59:48.149632+0000","last_unstale":"2024-04-26T21:59:49.117969+0000","last_undegraded":"2024-04-26T21:59:49.117969+0000","last_fullsized":"2024-04-26T21:59:49.117969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:05.263939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720074+0000","last_change":"2024-04-26T21:59:43.038322+0000","last_active":"2024-04-26T21:59:50.720074+0000","last_peered":"2024-04-26T21:59:50.720074+0000","last_clean":"2024-04-26T21:59:50.720074+0000","last_became_active":"2024-04-26T21:59:43.037968+0000","last_became_peered":"2024-04-26T21:59:43.037968+0000","last_unstale":"2024-04-26T21:59:50.720074+0000","last_undegraded":"2024-04-26T21:59:50.720074+0000","last_fullsized":"2024-04-26T21:59:50.720074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:21.265367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720035+0000","last_change":"2024-04-26T21:59:44.054970+0000","last_active":"2024-04-26T21:59:50.720035+0000","last_peered":"2024-04-26T21:59:50.720035+0000","last_clean":"2024-04-26T21:59:50.720035+0000","last_became_active":"2024-04-26T21:59:44.054599+0000","last_became_peered":"2024-04-26T21:59:44.054599+0000","last_unstale":"2024-04-26T21:59:50.720035+0000","last_undegraded":"2024-04-26T21:59:50.720035+0000","last_fullsized":"2024-04-26T21:59:50.720035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:22.048544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117821+0000","last_change":"2024-04-26T21:59:48.136704+0000","last_active":"2024-04-26T21:59:49.117821+0000","last_peered":"2024-04-26T21:59:49.117821+0000","last_clean":"2024-04-26T21:59:49.117821+0000","last_became_active":"2024-04-26T21:59:48.136276+0000","last_became_peered":"2024-04-26T21:59:48.136276+0000","last_unstale":"2024-04-26T21:59:49.117821+0000","last_undegraded":"2024-04-26T21:59:49.117821+0000","last_fullsized":"2024-04-26T21:59:49.117821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:05.608497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719799+0000","last_change":"2024-04-26T21:59:43.031556+0000","last_active":"2024-04-26T21:59:50.719799+0000","last_peered":"2024-04-26T21:59:50.719799+0000","last_clean":"2024-04-26T21:59:50.719799+0000","last_became_active":"2024-04-26T21:59:43.031216+0000","last_became_peered":"2024-04-26T21:59:43.031216+0000","last_unstale":"2024-04-26T21:59:50.719799+0000","last_undegraded":"2024-04-26T21:59:50.719799+0000","last_fullsized":"2024-04-26T21:59:50.719799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:51.675630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120287+0000","last_change":"2024-04-26T21:59:44.048291+0000","last_active":"2024-04-26T21:59:49.120287+0000","last_peered":"2024-04-26T21:59:49.120287+0000","last_clean":"2024-04-26T21:59:49.120287+0000","last_became_active":"2024-04-26T21:59:44.047983+0000","last_became_peered":"2024-04-26T21:59:44.047983+0000","last_unstale":"2024-04-26T21:59:49.120287+0000","last_undegraded":"2024-04-26T21:59:49.120287+0000","last_fullsized":"2024-04-26T21:59:49.120287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:17.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313826+0000","last_change":"2024-04-26T21:59:48.129287+0000","last_active":"2024-04-26T21:59:50.313826+0000","last_peered":"2024-04-26T21:59:50.313826+0000","last_clean":"2024-04-26T21:59:50.313826+0000","last_became_active":"2024-04-26T21:59:48.128789+0000","last_became_peered":"2024-04-26T21:59:48.128789+0000","last_unstale":"2024-04-26T21:59:50.313826+0000","last_undegraded":"2024-04-26T21:59:50.313826+0000","last_fullsized":"2024-04-26T21:59:50.313826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:46.495215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120397+0000","last_change":"2024-04-26T21:59:43.034634+0000","last_active":"2024-04-26T21:59:49.120397+0000","last_peered":"2024-04-26T21:59:49.120397+0000","last_clean":"2024-04-26T21:59:49.120397+0000","last_became_active":"2024-04-26T21:59:43.034085+0000","last_became_peered":"2024-04-26T21:59:43.034085+0000","last_unstale":"2024-04-26T21:59:49.120397+0000","last_undegraded":"2024-04-26T21:59:49.120397+0000","last_fullsized":"2024-04-26T21:59:49.120397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:35.415892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719745+0000","last_change":"2024-04-26T21:59:44.058077+0000","last_active":"2024-04-26T21:59:50.719745+0000","last_peered":"2024-04-26T21:59:50.719745+0000","last_clean":"2024-04-26T21:59:50.719745+0000","last_became_active":"2024-04-26T21:59:44.057720+0000","last_became_peered":"2024-04-26T21:59:44.057720+0000","last_unstale":"2024-04-26T21:59:50.719745+0000","last_undegraded":"2024-04-26T21:59:50.719745+0000","last_fullsized":"2024-04-26T21:59:50.719745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:56.889733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313754+0000","last_change":"2024-04-26T21:59:48.149256+0000","last_active":"2024-04-26T21:59:50.313754+0000","last_peered":"2024-04-26T21:59:50.313754+0000","last_clean":"2024-04-26T21:59:50.313754+0000","last_became_active":"2024-04-26T21:59:48.148832+0000","last_became_peered":"2024-04-26T21:59:48.148832+0000","last_unstale":"2024-04-26T21:59:50.313754+0000","last_undegraded":"2024-04-26T21:59:50.313754+0000","last_fullsized":"2024-04-26T21:59:50.313754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:46.120634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388794+0000","last_change":"2024-04-26T21:59:43.050527+0000","last_active":"2024-04-26T21:59:50.388794+0000","last_peered":"2024-04-26T21:59:50.388794+0000","last_clean":"2024-04-26T21:59:50.388794+0000","last_became_active":"2024-04-26T21:59:43.050168+0000","last_became_peered":"2024-04-26T21:59:43.050168+0000","last_unstale":"2024-04-26T21:59:50.388794+0000","last_undegraded":"2024-04-26T21:59:50.388794+0000","last_fullsized":"2024-04-26T21:59:50.388794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:45.484392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120735+0000","last_change":"2024-04-26T21:59:44.063479+0000","last_active":"2024-04-26T21:59:49.120735+0000","last_peered":"2024-04-26T21:59:49.120735+0000","last_clean":"2024-04-26T21:59:49.120735+0000","last_became_active":"2024-04-26T21:59:44.063205+0000","last_became_peered":"2024-04-26T21:59:44.063205+0000","last_unstale":"2024-04-26T21:59:49.120735+0000","last_undegraded":"2024-04-26T21:59:49.120735+0000","last_fullsized":"2024-04-26T21:59:49.120735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:07.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149596+0000","last_change":"2024-04-26T21:59:48.155383+0000","last_active":"2024-04-26T21:59:51.149596+0000","last_peered":"2024-04-26T21:59:51.149596+0000","last_clean":"2024-04-26T21:59:51.149596+0000","last_became_active":"2024-04-26T21:59:48.154948+0000","last_became_peered":"2024-04-26T21:59:48.154948+0000","last_unstale":"2024-04-26T21:59:51.149596+0000","last_undegraded":"2024-04-26T21:59:51.149596+0000","last_fullsized":"2024-04-26T21:59:51.149596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:25.874999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722149+0000","last_change":"2024-04-26T21:59:43.061828+0000","last_active":"2024-04-26T21:59:50.722149+0000","last_peered":"2024-04-26T21:59:50.722149+0000","last_clean":"2024-04-26T21:59:50.722149+0000","last_became_active":"2024-04-26T21:59:43.061440+0000","last_became_peered":"2024-04-26T21:59:43.061440+0000","last_unstale":"2024-04-26T21:59:50.722149+0000","last_undegraded":"2024-04-26T21:59:50.722149+0000","last_fullsized":"2024-04-26T21:59:50.722149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:17.964410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120513+0000","last_change":"2024-04-26T21:59:44.056947+0000","last_active":"2024-04-26T21:59:49.120513+0000","last_peered":"2024-04-26T21:59:49.120513+0000","last_clean":"2024-04-26T21:59:49.120513+0000","last_became_active":"2024-04-26T21:59:44.056537+0000","last_became_peered":"2024-04-26T21:59:44.056537+0000","last_unstale":"2024-04-26T21:59:49.120513+0000","last_undegraded":"2024-04-26T21:59:49.120513+0000","last_fullsized":"2024-04-26T21:59:49.120513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:19.601355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324467+0000","last_change":"2024-04-26T21:59:48.141476+0000","last_active":"2024-04-26T21:59:50.324467+0000","last_peered":"2024-04-26T21:59:50.324467+0000","last_clean":"2024-04-26T21:59:50.324467+0000","last_became_active":"2024-04-26T21:59:48.141028+0000","last_became_peered":"2024-04-26T21:59:48.141028+0000","last_unstale":"2024-04-26T21:59:50.324467+0000","last_undegraded":"2024-04-26T21:59:50.324467+0000","last_fullsized":"2024-04-26T21:59:50.324467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:36.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119924+0000","last_change":"2024-04-26T21:59:43.035587+0000","last_active":"2024-04-26T21:59:49.119924+0000","last_peered":"2024-04-26T21:59:49.119924+0000","last_clean":"2024-04-26T21:59:49.119924+0000","last_became_active":"2024-04-26T21:59:43.035102+0000","last_became_peered":"2024-04-26T21:59:43.035102+0000","last_unstale":"2024-04-26T21:59:49.119924+0000","last_undegraded":"2024-04-26T21:59:49.119924+0000","last_fullsized":"2024-04-26T21:59:49.119924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:52.817321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458214+0000","last_change":"2024-04-26T21:59:44.062002+0000","last_active":"2024-04-26T21:59:49.458214+0000","last_peered":"2024-04-26T21:59:49.458214+0000","last_clean":"2024-04-26T21:59:49.458214+0000","last_became_active":"2024-04-26T21:59:44.058419+0000","last_became_peered":"2024-04-26T21:59:44.058419+0000","last_unstale":"2024-04-26T21:59:49.458214+0000","last_undegraded":"2024-04-26T21:59:49.458214+0000","last_fullsized":"2024-04-26T21:59:49.458214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:48.698093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120447+0000","last_change":"2024-04-26T21:59:48.134540+0000","last_active":"2024-04-26T21:59:49.120447+0000","last_peered":"2024-04-26T21:59:49.120447+0000","last_clean":"2024-04-26T21:59:49.120447+0000","last_became_active":"2024-04-26T21:59:48.133149+0000","last_became_peered":"2024-04-26T21:59:48.133149+0000","last_unstale":"2024-04-26T21:59:49.120447+0000","last_undegraded":"2024-04-26T21:59:49.120447+0000","last_fullsized":"2024-04-26T21:59:49.120447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:18.861332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389201+0000","last_change":"2024-04-26T21:59:43.053360+0000","last_active":"2024-04-26T21:59:50.389201+0000","last_peered":"2024-04-26T21:59:50.389201+0000","last_clean":"2024-04-26T21:59:50.389201+0000","last_became_active":"2024-04-26T21:59:43.053113+0000","last_became_peered":"2024-04-26T21:59:43.053113+0000","last_unstale":"2024-04-26T21:59:50.389201+0000","last_undegraded":"2024-04-26T21:59:50.389201+0000","last_fullsized":"2024-04-26T21:59:50.389201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:38.464573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720168+0000","last_change":"2024-04-26T21:59:44.060319+0000","last_active":"2024-04-26T21:59:50.720168+0000","last_peered":"2024-04-26T21:59:50.720168+0000","last_clean":"2024-04-26T21:59:50.720168+0000","last_became_active":"2024-04-26T21:59:44.059947+0000","last_became_peered":"2024-04-26T21:59:44.059947+0000","last_unstale":"2024-04-26T21:59:50.720168+0000","last_undegraded":"2024-04-26T21:59:50.720168+0000","last_fullsized":"2024-04-26T21:59:50.720168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:01.851669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458423+0000","last_change":"2024-04-26T21:59:48.148358+0000","last_active":"2024-04-26T21:59:49.458423+0000","last_peered":"2024-04-26T21:59:49.458423+0000","last_clean":"2024-04-26T21:59:49.458423+0000","last_became_active":"2024-04-26T21:59:48.143520+0000","last_became_peered":"2024-04-26T21:59:48.143520+0000","last_unstale":"2024-04-26T21:59:49.458423+0000","last_undegraded":"2024-04-26T21:59:49.458423+0000","last_fullsized":"2024-04-26T21:59:49.458423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:23.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661626+0000","last_change":"2024-04-26T21:59:43.037153+0000","last_active":"2024-04-26T21:59:50.661626+0000","last_peered":"2024-04-26T21:59:50.661626+0000","last_clean":"2024-04-26T21:59:50.661626+0000","last_became_active":"2024-04-26T21:59:43.036816+0000","last_became_peered":"2024-04-26T21:59:43.036816+0000","last_unstale":"2024-04-26T21:59:50.661626+0000","last_undegraded":"2024-04-26T21:59:50.661626+0000","last_fullsized":"2024-04-26T21:59:50.661626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:44.655304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324672+0000","last_change":"2024-04-26T21:59:44.047947+0000","last_active":"2024-04-26T21:59:50.324672+0000","last_peered":"2024-04-26T21:59:50.324672+0000","last_clean":"2024-04-26T21:59:50.324672+0000","last_became_active":"2024-04-26T21:59:44.047586+0000","last_became_peered":"2024-04-26T21:59:44.047586+0000","last_unstale":"2024-04-26T21:59:50.324672+0000","last_undegraded":"2024-04-26T21:59:50.324672+0000","last_fullsized":"2024-04-26T21:59:50.324672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:49.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118267+0000","last_change":"2024-04-26T21:59:48.150787+0000","last_active":"2024-04-26T21:59:49.118267+0000","last_peered":"2024-04-26T21:59:49.118267+0000","last_clean":"2024-04-26T21:59:49.118267+0000","last_became_active":"2024-04-26T21:59:48.150004+0000","last_became_peered":"2024-04-26T21:59:48.150004+0000","last_unstale":"2024-04-26T21:59:49.118267+0000","last_undegraded":"2024-04-26T21:59:49.118267+0000","last_fullsized":"2024-04-26T21:59:49.118267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:19.657837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120367+0000","last_change":"2024-04-26T21:59:43.050931+0000","last_active":"2024-04-26T21:59:49.120367+0000","last_peered":"2024-04-26T21:59:49.120367+0000","last_clean":"2024-04-26T21:59:49.120367+0000","last_became_active":"2024-04-26T21:59:43.050508+0000","last_became_peered":"2024-04-26T21:59:43.050508+0000","last_unstale":"2024-04-26T21:59:49.120367+0000","last_undegraded":"2024-04-26T21:59:49.120367+0000","last_fullsized":"2024-04-26T21:59:49.120367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:08.378649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720132+0000","last_change":"2024-04-26T21:59:44.039078+0000","last_active":"2024-04-26T21:59:50.720132+0000","last_peered":"2024-04-26T21:59:50.720132+0000","last_clean":"2024-04-26T21:59:50.720132+0000","last_became_active":"2024-04-26T21:59:44.038734+0000","last_became_peered":"2024-04-26T21:59:44.038734+0000","last_unstale":"2024-04-26T21:59:50.720132+0000","last_undegraded":"2024-04-26T21:59:50.720132+0000","last_fullsized":"2024-04-26T21:59:50.720132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:07.551134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458575+0000","last_change":"2024-04-26T21:59:48.150464+0000","last_active":"2024-04-26T21:59:49.458575+0000","last_peered":"2024-04-26T21:59:49.458575+0000","last_clean":"2024-04-26T21:59:49.458575+0000","last_became_active":"2024-04-26T21:59:48.150095+0000","last_became_peered":"2024-04-26T21:59:48.150095+0000","last_unstale":"2024-04-26T21:59:49.458575+0000","last_undegraded":"2024-04-26T21:59:49.458575+0000","last_fullsized":"2024-04-26T21:59:49.458575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:53.152840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:00:09.250045+0000","last_change":"2024-04-26T21:59:43.038751+0000","last_active":"2024-04-26T22:00:09.250045+0000","last_peered":"2024-04-26T22:00:09.250045+0000","last_clean":"2024-04-26T22:00:09.250045+0000","last_became_active":"2024-04-26T21:59:43.038337+0000","last_became_peered":"2024-04-26T21:59:43.038337+0000","last_unstale":"2024-04-26T22:00:09.250045+0000","last_undegraded":"2024-04-26T22:00:09.250045+0000","last_fullsized":"2024-04-26T22:00:09.250045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.830357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719914+0000","last_change":"2024-04-26T21:59:44.040170+0000","last_active":"2024-04-26T21:59:50.719914+0000","last_peered":"2024-04-26T21:59:50.719914+0000","last_clean":"2024-04-26T21:59:50.719914+0000","last_became_active":"2024-04-26T21:59:44.039713+0000","last_became_peered":"2024-04-26T21:59:44.039713+0000","last_unstale":"2024-04-26T21:59:50.719914+0000","last_undegraded":"2024-04-26T21:59:50.719914+0000","last_fullsized":"2024-04-26T21:59:50.719914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:49.036612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121087+0000","last_change":"2024-04-26T21:59:48.140066+0000","last_active":"2024-04-26T21:59:49.121087+0000","last_peered":"2024-04-26T21:59:49.121087+0000","last_clean":"2024-04-26T21:59:49.121087+0000","last_became_active":"2024-04-26T21:59:48.139748+0000","last_became_peered":"2024-04-26T21:59:48.139748+0000","last_unstale":"2024-04-26T21:59:49.121087+0000","last_undegraded":"2024-04-26T21:59:49.121087+0000","last_fullsized":"2024-04-26T21:59:49.121087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:12.280574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313643+0000","last_change":"2024-04-26T21:59:43.039994+0000","last_active":"2024-04-26T21:59:50.313643+0000","last_peered":"2024-04-26T21:59:50.313643+0000","last_clean":"2024-04-26T21:59:50.313643+0000","last_became_active":"2024-04-26T21:59:43.039671+0000","last_became_peered":"2024-04-26T21:59:43.039671+0000","last_unstale":"2024-04-26T21:59:50.313643+0000","last_undegraded":"2024-04-26T21:59:50.313643+0000","last_fullsized":"2024-04-26T21:59:50.313643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:59.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722856+0000","last_change":"2024-04-26T21:59:44.065515+0000","last_active":"2024-04-26T21:59:50.722856+0000","last_peered":"2024-04-26T21:59:50.722856+0000","last_clean":"2024-04-26T21:59:50.722856+0000","last_became_active":"2024-04-26T21:59:44.065120+0000","last_became_peered":"2024-04-26T21:59:44.065120+0000","last_unstale":"2024-04-26T21:59:50.722856+0000","last_undegraded":"2024-04-26T21:59:50.722856+0000","last_fullsized":"2024-04-26T21:59:50.722856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:09.709198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149237+0000","last_change":"2024-04-26T21:59:48.149372+0000","last_active":"2024-04-26T21:59:51.149237+0000","last_peered":"2024-04-26T21:59:51.149237+0000","last_clean":"2024-04-26T21:59:51.149237+0000","last_became_active":"2024-04-26T21:59:48.148920+0000","last_became_peered":"2024-04-26T21:59:48.148920+0000","last_unstale":"2024-04-26T21:59:51.149237+0000","last_undegraded":"2024-04-26T21:59:51.149237+0000","last_fullsized":"2024-04-26T21:59:51.149237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:46.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120451+0000","last_change":"2024-04-26T21:59:43.040576+0000","last_active":"2024-04-26T21:59:49.120451+0000","last_peered":"2024-04-26T21:59:49.120451+0000","last_clean":"2024-04-26T21:59:49.120451+0000","last_became_active":"2024-04-26T21:59:43.040217+0000","last_became_peered":"2024-04-26T21:59:43.040217+0000","last_unstale":"2024-04-26T21:59:49.120451+0000","last_undegraded":"2024-04-26T21:59:49.120451+0000","last_fullsized":"2024-04-26T21:59:49.120451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:58.860607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661528+0000","last_change":"2024-04-26T21:59:44.067746+0000","last_active":"2024-04-26T21:59:50.661528+0000","last_peered":"2024-04-26T21:59:50.661528+0000","last_clean":"2024-04-26T21:59:50.661528+0000","last_became_active":"2024-04-26T21:59:44.067434+0000","last_became_peered":"2024-04-26T21:59:44.067434+0000","last_unstale":"2024-04-26T21:59:50.661528+0000","last_undegraded":"2024-04-26T21:59:50.661528+0000","last_fullsized":"2024-04-26T21:59:50.661528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:45.304899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459553+0000","last_change":"2024-04-26T21:59:48.157380+0000","last_active":"2024-04-26T21:59:49.459553+0000","last_peered":"2024-04-26T21:59:49.459553+0000","last_clean":"2024-04-26T21:59:49.459553+0000","last_became_active":"2024-04-26T21:59:48.155887+0000","last_became_peered":"2024-04-26T21:59:48.155887+0000","last_unstale":"2024-04-26T21:59:49.459553+0000","last_undegraded":"2024-04-26T21:59:49.459553+0000","last_fullsized":"2024-04-26T21:59:49.459553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.339191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117947+0000","last_change":"2024-04-26T21:59:43.036773+0000","last_active":"2024-04-26T21:59:49.117947+0000","last_peered":"2024-04-26T21:59:49.117947+0000","last_clean":"2024-04-26T21:59:49.117947+0000","last_became_active":"2024-04-26T21:59:43.036384+0000","last_became_peered":"2024-04-26T21:59:43.036384+0000","last_unstale":"2024-04-26T21:59:49.117947+0000","last_undegraded":"2024-04-26T21:59:49.117947+0000","last_fullsized":"2024-04-26T21:59:49.117947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:33.841309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661381+0000","last_change":"2024-04-26T21:59:44.043546+0000","last_active":"2024-04-26T21:59:50.661381+0000","last_peered":"2024-04-26T21:59:50.661381+0000","last_clean":"2024-04-26T21:59:50.661381+0000","last_became_active":"2024-04-26T21:59:44.043221+0000","last_became_peered":"2024-04-26T21:59:44.043221+0000","last_unstale":"2024-04-26T21:59:50.661381+0000","last_undegraded":"2024-04-26T21:59:50.661381+0000","last_fullsized":"2024-04-26T21:59:50.661381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.436179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723618+0000","last_change":"2024-04-26T21:59:48.156690+0000","last_active":"2024-04-26T21:59:50.723618+0000","last_peered":"2024-04-26T21:59:50.723618+0000","last_clean":"2024-04-26T21:59:50.723618+0000","last_became_active":"2024-04-26T21:59:48.155067+0000","last_became_peered":"2024-04-26T21:59:48.155067+0000","last_unstale":"2024-04-26T21:59:50.723618+0000","last_undegraded":"2024-04-26T21:59:50.723618+0000","last_fullsized":"2024-04-26T21:59:50.723618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:39.366540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719457+0000","last_change":"2024-04-26T21:59:43.050964+0000","last_active":"2024-04-26T21:59:50.719457+0000","last_peered":"2024-04-26T21:59:50.719457+0000","last_clean":"2024-04-26T21:59:50.719457+0000","last_became_active":"2024-04-26T21:59:43.050622+0000","last_became_peered":"2024-04-26T21:59:43.050622+0000","last_unstale":"2024-04-26T21:59:50.719457+0000","last_undegraded":"2024-04-26T21:59:50.719457+0000","last_fullsized":"2024-04-26T21:59:50.719457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:18.246708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120090+0000","last_change":"2024-04-26T21:59:44.044126+0000","last_active":"2024-04-26T21:59:49.120090+0000","last_peered":"2024-04-26T21:59:49.120090+0000","last_clean":"2024-04-26T21:59:49.120090+0000","last_became_active":"2024-04-26T21:59:44.043800+0000","last_became_peered":"2024-04-26T21:59:44.043800+0000","last_unstale":"2024-04-26T21:59:49.120090+0000","last_undegraded":"2024-04-26T21:59:49.120090+0000","last_fullsized":"2024-04-26T21:59:49.120090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:46.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120529+0000","last_change":"2024-04-26T21:59:48.134868+0000","last_active":"2024-04-26T21:59:49.120529+0000","last_peered":"2024-04-26T21:59:49.120529+0000","last_clean":"2024-04-26T21:59:49.120529+0000","last_became_active":"2024-04-26T21:59:48.134545+0000","last_became_peered":"2024-04-26T21:59:48.134545+0000","last_unstale":"2024-04-26T21:59:49.120529+0000","last_undegraded":"2024-04-26T21:59:49.120529+0000","last_fullsized":"2024-04-26T21:59:49.120529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:59.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720077+0000","last_change":"2024-04-26T21:59:43.038912+0000","last_active":"2024-04-26T21:59:50.720077+0000","last_peered":"2024-04-26T21:59:50.720077+0000","last_clean":"2024-04-26T21:59:50.720077+0000","last_became_active":"2024-04-26T21:59:43.038597+0000","last_became_peered":"2024-04-26T21:59:43.038597+0000","last_unstale":"2024-04-26T21:59:50.720077+0000","last_undegraded":"2024-04-26T21:59:50.720077+0000","last_fullsized":"2024-04-26T21:59:50.720077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:37.087595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149414+0000","last_change":"2024-04-26T21:59:44.043297+0000","last_active":"2024-04-26T21:59:51.149414+0000","last_peered":"2024-04-26T21:59:51.149414+0000","last_clean":"2024-04-26T21:59:51.149414+0000","last_became_active":"2024-04-26T21:59:44.042952+0000","last_became_peered":"2024-04-26T21:59:44.042952+0000","last_unstale":"2024-04-26T21:59:51.149414+0000","last_undegraded":"2024-04-26T21:59:51.149414+0000","last_fullsized":"2024-04-26T21:59:51.149414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:18.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719851+0000","last_change":"2024-04-26T21:59:48.155119+0000","last_active":"2024-04-26T21:59:50.719851+0000","last_peered":"2024-04-26T21:59:50.719851+0000","last_clean":"2024-04-26T21:59:50.719851+0000","last_became_active":"2024-04-26T21:59:48.154593+0000","last_became_peered":"2024-04-26T21:59:48.154593+0000","last_unstale":"2024-04-26T21:59:50.719851+0000","last_undegraded":"2024-04-26T21:59:50.719851+0000","last_fullsized":"2024-04-26T21:59:50.719851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:01.896610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459017+0000","last_change":"2024-04-26T21:59:43.067817+0000","last_active":"2024-04-26T21:59:49.459017+0000","last_peered":"2024-04-26T21:59:49.459017+0000","last_clean":"2024-04-26T21:59:49.459017+0000","last_became_active":"2024-04-26T21:59:43.067450+0000","last_became_peered":"2024-04-26T21:59:43.067450+0000","last_unstale":"2024-04-26T21:59:49.459017+0000","last_undegraded":"2024-04-26T21:59:49.459017+0000","last_fullsized":"2024-04-26T21:59:49.459017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:29.425903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389031+0000","last_change":"2024-04-26T21:59:44.055085+0000","last_active":"2024-04-26T21:59:50.389031+0000","last_peered":"2024-04-26T21:59:50.389031+0000","last_clean":"2024-04-26T21:59:50.389031+0000","last_became_active":"2024-04-26T21:59:44.054627+0000","last_became_peered":"2024-04-26T21:59:44.054627+0000","last_unstale":"2024-04-26T21:59:50.389031+0000","last_undegraded":"2024-04-26T21:59:50.389031+0000","last_fullsized":"2024-04-26T21:59:50.389031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.234287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722992+0000","last_change":"2024-04-26T21:59:48.147333+0000","last_active":"2024-04-26T21:59:50.722992+0000","last_peered":"2024-04-26T21:59:50.722992+0000","last_clean":"2024-04-26T21:59:50.722992+0000","last_became_active":"2024-04-26T21:59:48.146926+0000","last_became_peered":"2024-04-26T21:59:48.146926+0000","last_unstale":"2024-04-26T21:59:50.722992+0000","last_undegraded":"2024-04-26T21:59:50.722992+0000","last_fullsized":"2024-04-26T21:59:50.722992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661244+0000","last_change":"2024-04-26T21:59:43.050434+0000","last_active":"2024-04-26T21:59:50.661244+0000","last_peered":"2024-04-26T21:59:50.661244+0000","last_clean":"2024-04-26T21:59:50.661244+0000","last_became_active":"2024-04-26T21:59:43.050096+0000","last_became_peered":"2024-04-26T21:59:43.050096+0000","last_unstale":"2024-04-26T21:59:50.661244+0000","last_undegraded":"2024-04-26T21:59:50.661244+0000","last_fullsized":"2024-04-26T21:59:50.661244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:22.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388887+0000","last_change":"2024-04-26T21:59:44.048302+0000","last_active":"2024-04-26T21:59:50.388887+0000","last_peered":"2024-04-26T21:59:50.388887+0000","last_clean":"2024-04-26T21:59:50.388887+0000","last_became_active":"2024-04-26T21:59:44.047898+0000","last_became_peered":"2024-04-26T21:59:44.047898+0000","last_unstale":"2024-04-26T21:59:50.388887+0000","last_undegraded":"2024-04-26T21:59:50.388887+0000","last_fullsized":"2024-04-26T21:59:50.388887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:55.718558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723553+0000","last_change":"2024-04-26T21:59:48.148025+0000","last_active":"2024-04-26T21:59:50.723553+0000","last_peered":"2024-04-26T21:59:50.723553+0000","last_clean":"2024-04-26T21:59:50.723553+0000","last_became_active":"2024-04-26T21:59:48.147065+0000","last_became_peered":"2024-04-26T21:59:48.147065+0000","last_unstale":"2024-04-26T21:59:50.723553+0000","last_undegraded":"2024-04-26T21:59:50.723553+0000","last_fullsized":"2024-04-26T21:59:50.723553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:23.932501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120192+0000","last_change":"2024-04-26T21:59:44.045186+0000","last_active":"2024-04-26T21:59:49.120192+0000","last_peered":"2024-04-26T21:59:49.120192+0000","last_clean":"2024-04-26T21:59:49.120192+0000","last_became_active":"2024-04-26T21:59:44.044761+0000","last_became_peered":"2024-04-26T21:59:44.044761+0000","last_unstale":"2024-04-26T21:59:49.120192+0000","last_undegraded":"2024-04-26T21:59:49.120192+0000","last_fullsized":"2024-04-26T21:59:49.120192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:32.744412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722062+0000","last_change":"2024-04-26T21:59:43.068695+0000","last_active":"2024-04-26T21:59:50.722062+0000","last_peered":"2024-04-26T21:59:50.722062+0000","last_clean":"2024-04-26T21:59:50.722062+0000","last_became_active":"2024-04-26T21:59:43.068210+0000","last_became_peered":"2024-04-26T21:59:43.068210+0000","last_unstale":"2024-04-26T21:59:50.722062+0000","last_undegraded":"2024-04-26T21:59:50.722062+0000","last_fullsized":"2024-04-26T21:59:50.722062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:19.914872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661647+0000","last_change":"2024-04-26T21:59:48.154253+0000","last_active":"2024-04-26T21:59:50.661647+0000","last_peered":"2024-04-26T21:59:50.661647+0000","last_clean":"2024-04-26T21:59:50.661647+0000","last_became_active":"2024-04-26T21:59:48.153877+0000","last_became_peered":"2024-04-26T21:59:48.153877+0000","last_unstale":"2024-04-26T21:59:50.661647+0000","last_undegraded":"2024-04-26T21:59:50.661647+0000","last_fullsized":"2024-04-26T21:59:50.661647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:28.163306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120994+0000","last_change":"2024-04-26T21:59:44.052732+0000","last_active":"2024-04-26T21:59:49.120994+0000","last_peered":"2024-04-26T21:59:49.120994+0000","last_clean":"2024-04-26T21:59:49.120994+0000","last_became_active":"2024-04-26T21:59:44.051202+0000","last_became_peered":"2024-04-26T21:59:44.051202+0000","last_unstale":"2024-04-26T21:59:49.120994+0000","last_undegraded":"2024-04-26T21:59:49.120994+0000","last_fullsized":"2024-04-26T21:59:49.120994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:20.887239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118164+0000","last_change":"2024-04-26T21:59:43.054776+0000","last_active":"2024-04-26T21:59:49.118164+0000","last_peered":"2024-04-26T21:59:49.118164+0000","last_clean":"2024-04-26T21:59:49.118164+0000","last_became_active":"2024-04-26T21:59:43.054410+0000","last_became_peered":"2024-04-26T21:59:43.054410+0000","last_unstale":"2024-04-26T21:59:49.118164+0000","last_undegraded":"2024-04-26T21:59:49.118164+0000","last_fullsized":"2024-04-26T21:59:49.118164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:11.675134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661021+0000","last_change":"2024-04-26T21:58:22.546842+0000","last_active":"2024-04-26T21:59:50.661021+0000","last_peered":"2024-04-26T21:59:50.661021+0000","last_clean":"2024-04-26T21:59:50.661021+0000","last_became_active":"2024-04-26T21:58:22.546448+0000","last_became_peered":"2024-04-26T21:58:22.546448+0000","last_unstale":"2024-04-26T21:59:50.661021+0000","last_undegraded":"2024-04-26T21:59:50.661021+0000","last_fullsized":"2024-04-26T21:59:50.661021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_clean_scrub_stamp":"2024-04-26T21:54:29.502062+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:31.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120544+0000","last_change":"2024-04-26T21:59:48.152986+0000","last_active":"2024-04-26T21:59:49.120544+0000","last_peered":"2024-04-26T21:59:49.120544+0000","last_clean":"2024-04-26T21:59:49.120544+0000","last_became_active":"2024-04-26T21:59:48.152606+0000","last_became_peered":"2024-04-26T21:59:48.152606+0000","last_unstale":"2024-04-26T21:59:49.120544+0000","last_undegraded":"2024-04-26T21:59:49.120544+0000","last_fullsized":"2024-04-26T21:59:49.120544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.721974+0000","last_change":"2024-04-26T21:59:44.060086+0000","last_active":"2024-04-26T21:59:50.721974+0000","last_peered":"2024-04-26T21:59:50.721974+0000","last_clean":"2024-04-26T21:59:50.721974+0000","last_became_active":"2024-04-26T21:59:44.059742+0000","last_became_peered":"2024-04-26T21:59:44.059742+0000","last_unstale":"2024-04-26T21:59:50.721974+0000","last_undegraded":"2024-04-26T21:59:50.721974+0000","last_fullsized":"2024-04-26T21:59:50.721974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.678769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661351+0000","last_change":"2024-04-26T21:59:43.034726+0000","last_active":"2024-04-26T21:59:50.661351+0000","last_peered":"2024-04-26T21:59:50.661351+0000","last_clean":"2024-04-26T21:59:50.661351+0000","last_became_active":"2024-04-26T21:59:43.034407+0000","last_became_peered":"2024-04-26T21:59:43.034407+0000","last_unstale":"2024-04-26T21:59:50.661351+0000","last_undegraded":"2024-04-26T21:59:50.661351+0000","last_fullsized":"2024-04-26T21:59:50.661351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:37.722860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149321+0000","last_change":"2024-04-26T21:59:48.136044+0000","last_active":"2024-04-26T21:59:51.149321+0000","last_peered":"2024-04-26T21:59:51.149321+0000","last_clean":"2024-04-26T21:59:51.149321+0000","last_became_active":"2024-04-26T21:59:48.135619+0000","last_became_peered":"2024-04-26T21:59:48.135619+0000","last_unstale":"2024-04-26T21:59:51.149321+0000","last_undegraded":"2024-04-26T21:59:51.149321+0000","last_fullsized":"2024-04-26T21:59:51.149321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:24.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719923+0000","last_change":"2024-04-26T21:59:43.059149+0000","last_active":"2024-04-26T21:59:50.719923+0000","last_peered":"2024-04-26T21:59:50.719923+0000","last_clean":"2024-04-26T21:59:50.719923+0000","last_became_active":"2024-04-26T21:59:43.058816+0000","last_became_peered":"2024-04-26T21:59:43.058816+0000","last_unstale":"2024-04-26T21:59:50.719923+0000","last_undegraded":"2024-04-26T21:59:50.719923+0000","last_fullsized":"2024-04-26T21:59:50.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:04.441843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723197+0000","last_change":"2024-04-26T21:59:44.063544+0000","last_active":"2024-04-26T21:59:50.723197+0000","last_peered":"2024-04-26T21:59:50.723197+0000","last_clean":"2024-04-26T21:59:50.723197+0000","last_became_active":"2024-04-26T21:59:44.063179+0000","last_became_peered":"2024-04-26T21:59:44.063179+0000","last_unstale":"2024-04-26T21:59:50.723197+0000","last_undegraded":"2024-04-26T21:59:50.723197+0000","last_fullsized":"2024-04-26T21:59:50.723197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:44.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120312+0000","last_change":"2024-04-26T21:59:48.131695+0000","last_active":"2024-04-26T21:59:49.120312+0000","last_peered":"2024-04-26T21:59:49.120312+0000","last_clean":"2024-04-26T21:59:49.120312+0000","last_became_active":"2024-04-26T21:59:48.131246+0000","last_became_peered":"2024-04-26T21:59:48.131246+0000","last_unstale":"2024-04-26T21:59:49.120312+0000","last_undegraded":"2024-04-26T21:59:49.120312+0000","last_fullsized":"2024-04-26T21:59:49.120312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:53.128553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:00:11.079444+0000","last_change":"2024-04-26T21:59:43.067463+0000","last_active":"2024-04-26T22:00:11.079444+0000","last_peered":"2024-04-26T22:00:11.079444+0000","last_clean":"2024-04-26T22:00:11.079444+0000","last_became_active":"2024-04-26T21:59:43.067009+0000","last_became_peered":"2024-04-26T21:59:43.067009+0000","last_unstale":"2024-04-26T22:00:11.079444+0000","last_undegraded":"2024-04-26T22:00:11.079444+0000","last_fullsized":"2024-04-26T22:00:11.079444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:35.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122236+0000","last_change":"2024-04-26T21:59:44.054781+0000","last_active":"2024-04-26T21:59:49.122236+0000","last_peered":"2024-04-26T21:59:49.122236+0000","last_clean":"2024-04-26T21:59:49.122236+0000","last_became_active":"2024-04-26T21:59:44.049439+0000","last_became_peered":"2024-04-26T21:59:44.049439+0000","last_unstale":"2024-04-26T21:59:49.122236+0000","last_undegraded":"2024-04-26T21:59:49.122236+0000","last_fullsized":"2024-04-26T21:59:49.122236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:50.523835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313357+0000","last_change":"2024-04-26T21:59:48.127776+0000","last_active":"2024-04-26T21:59:50.313357+0000","last_peered":"2024-04-26T21:59:50.313357+0000","last_clean":"2024-04-26T21:59:50.313357+0000","last_became_active":"2024-04-26T21:59:48.127207+0000","last_became_peered":"2024-04-26T21:59:48.127207+0000","last_unstale":"2024-04-26T21:59:50.313357+0000","last_undegraded":"2024-04-26T21:59:50.313357+0000","last_fullsized":"2024-04-26T21:59:50.313357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:23.794371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118341+0000","last_change":"2024-04-26T21:59:43.050320+0000","last_active":"2024-04-26T21:59:49.118341+0000","last_peered":"2024-04-26T21:59:49.118341+0000","last_clean":"2024-04-26T21:59:49.118341+0000","last_became_active":"2024-04-26T21:59:43.049890+0000","last_became_peered":"2024-04-26T21:59:43.049890+0000","last_unstale":"2024-04-26T21:59:49.118341+0000","last_undegraded":"2024-04-26T21:59:49.118341+0000","last_fullsized":"2024-04-26T21:59:49.118341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:53.589776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324595+0000","last_change":"2024-04-26T21:59:44.057851+0000","last_active":"2024-04-26T21:59:50.324595+0000","last_peered":"2024-04-26T21:59:50.324595+0000","last_clean":"2024-04-26T21:59:50.324595+0000","last_became_active":"2024-04-26T21:59:44.057525+0000","last_became_peered":"2024-04-26T21:59:44.057525+0000","last_unstale":"2024-04-26T21:59:50.324595+0000","last_undegraded":"2024-04-26T21:59:50.324595+0000","last_fullsized":"2024-04-26T21:59:50.324595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:30.986251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118072+0000","last_change":"2024-04-26T21:59:48.154962+0000","last_active":"2024-04-26T21:59:49.118072+0000","last_peered":"2024-04-26T21:59:49.118072+0000","last_clean":"2024-04-26T21:59:49.118072+0000","last_became_active":"2024-04-26T21:59:48.154479+0000","last_became_peered":"2024-04-26T21:59:48.154479+0000","last_unstale":"2024-04-26T21:59:49.118072+0000","last_undegraded":"2024-04-26T21:59:49.118072+0000","last_fullsized":"2024-04-26T21:59:49.118072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:30.415432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719716+0000","last_change":"2024-04-26T21:59:43.051186+0000","last_active":"2024-04-26T21:59:50.719716+0000","last_peered":"2024-04-26T21:59:50.719716+0000","last_clean":"2024-04-26T21:59:50.719716+0000","last_became_active":"2024-04-26T21:59:43.050768+0000","last_became_peered":"2024-04-26T21:59:43.050768+0000","last_unstale":"2024-04-26T21:59:50.719716+0000","last_undegraded":"2024-04-26T21:59:50.719716+0000","last_fullsized":"2024-04-26T21:59:50.719716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.237229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719465+0000","last_change":"2024-04-26T21:59:44.039113+0000","last_active":"2024-04-26T21:59:50.719465+0000","last_peered":"2024-04-26T21:59:50.719465+0000","last_clean":"2024-04-26T21:59:50.719465+0000","last_became_active":"2024-04-26T21:59:44.038610+0000","last_became_peered":"2024-04-26T21:59:44.038610+0000","last_unstale":"2024-04-26T21:59:50.719465+0000","last_undegraded":"2024-04-26T21:59:50.719465+0000","last_fullsized":"2024-04-26T21:59:50.719465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:39.450603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.460041+0000","last_change":"2024-04-26T21:59:48.157058+0000","last_active":"2024-04-26T21:59:49.460041+0000","last_peered":"2024-04-26T21:59:49.460041+0000","last_clean":"2024-04-26T21:59:49.460041+0000","last_became_active":"2024-04-26T21:59:48.153897+0000","last_became_peered":"2024-04-26T21:59:48.153897+0000","last_unstale":"2024-04-26T21:59:49.460041+0000","last_undegraded":"2024-04-26T21:59:49.460041+0000","last_fullsized":"2024-04-26T21:59:49.460041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.044493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120593+0000","last_change":"2024-04-26T21:59:43.061101+0000","last_active":"2024-04-26T21:59:49.120593+0000","last_peered":"2024-04-26T21:59:49.120593+0000","last_clean":"2024-04-26T21:59:49.120593+0000","last_became_active":"2024-04-26T21:59:43.060762+0000","last_became_peered":"2024-04-26T21:59:43.060762+0000","last_unstale":"2024-04-26T21:59:49.120593+0000","last_undegraded":"2024-04-26T21:59:49.120593+0000","last_fullsized":"2024-04-26T21:59:49.120593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:07.961535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149273+0000","last_change":"2024-04-26T21:59:44.047437+0000","last_active":"2024-04-26T21:59:51.149273+0000","last_peered":"2024-04-26T21:59:51.149273+0000","last_clean":"2024-04-26T21:59:51.149273+0000","last_became_active":"2024-04-26T21:59:44.047127+0000","last_became_peered":"2024-04-26T21:59:44.047127+0000","last_unstale":"2024-04-26T21:59:51.149273+0000","last_undegraded":"2024-04-26T21:59:51.149273+0000","last_fullsized":"2024-04-26T21:59:51.149273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:12.259965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118210+0000","last_change":"2024-04-26T21:59:48.149822+0000","last_active":"2024-04-26T21:59:49.118210+0000","last_peered":"2024-04-26T21:59:49.118210+0000","last_clean":"2024-04-26T21:59:49.118210+0000","last_became_active":"2024-04-26T21:59:48.149409+0000","last_became_peered":"2024-04-26T21:59:48.149409+0000","last_unstale":"2024-04-26T21:59:49.118210+0000","last_undegraded":"2024-04-26T21:59:49.118210+0000","last_fullsized":"2024-04-26T21:59:49.118210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:14.161867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313388+0000","last_change":"2024-04-26T21:59:43.043901+0000","last_active":"2024-04-26T21:59:50.313388+0000","last_peered":"2024-04-26T21:59:50.313388+0000","last_clean":"2024-04-26T21:59:50.313388+0000","last_became_active":"2024-04-26T21:59:43.043572+0000","last_became_peered":"2024-04-26T21:59:43.043572+0000","last_unstale":"2024-04-26T21:59:50.313388+0000","last_undegraded":"2024-04-26T21:59:50.313388+0000","last_fullsized":"2024-04-26T21:59:50.313388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.107295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120801+0000","last_change":"2024-04-26T21:59:44.059451+0000","last_active":"2024-04-26T21:59:49.120801+0000","last_peered":"2024-04-26T21:59:49.120801+0000","last_clean":"2024-04-26T21:59:49.120801+0000","last_became_active":"2024-04-26T21:59:44.056771+0000","last_became_peered":"2024-04-26T21:59:44.056771+0000","last_unstale":"2024-04-26T21:59:49.120801+0000","last_undegraded":"2024-04-26T21:59:49.120801+0000","last_fullsized":"2024-04-26T21:59:49.120801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:53.641182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118177+0000","last_change":"2024-04-26T21:59:48.130222+0000","last_active":"2024-04-26T21:59:49.118177+0000","last_peered":"2024-04-26T21:59:49.118177+0000","last_clean":"2024-04-26T21:59:49.118177+0000","last_became_active":"2024-04-26T21:59:48.129379+0000","last_became_peered":"2024-04-26T21:59:48.129379+0000","last_unstale":"2024-04-26T21:59:49.118177+0000","last_undegraded":"2024-04-26T21:59:49.118177+0000","last_fullsized":"2024-04-26T21:59:49.118177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:26.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719646+0000","last_change":"2024-04-26T21:59:43.035503+0000","last_active":"2024-04-26T21:59:50.719646+0000","last_peered":"2024-04-26T21:59:50.719646+0000","last_clean":"2024-04-26T21:59:50.719646+0000","last_became_active":"2024-04-26T21:59:43.035180+0000","last_became_peered":"2024-04-26T21:59:43.035180+0000","last_unstale":"2024-04-26T21:59:50.719646+0000","last_undegraded":"2024-04-26T21:59:50.719646+0000","last_fullsized":"2024-04-26T21:59:50.719646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:45.696707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661312+0000","last_change":"2024-04-26T21:59:44.054444+0000","last_active":"2024-04-26T21:59:50.661312+0000","last_peered":"2024-04-26T21:59:50.661312+0000","last_clean":"2024-04-26T21:59:50.661312+0000","last_became_active":"2024-04-26T21:59:44.054035+0000","last_became_peered":"2024-04-26T21:59:44.054035+0000","last_unstale":"2024-04-26T21:59:50.661312+0000","last_undegraded":"2024-04-26T21:59:50.661312+0000","last_fullsized":"2024-04-26T21:59:50.661312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:04.905286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719879+0000","last_change":"2024-04-26T21:59:48.141551+0000","last_active":"2024-04-26T21:59:50.719879+0000","last_peered":"2024-04-26T21:59:50.719879+0000","last_clean":"2024-04-26T21:59:50.719879+0000","last_became_active":"2024-04-26T21:59:48.141166+0000","last_became_peered":"2024-04-26T21:59:48.141166+0000","last_unstale":"2024-04-26T21:59:50.719879+0000","last_undegraded":"2024-04-26T21:59:50.719879+0000","last_fullsized":"2024-04-26T21:59:50.719879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:38.810387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313994+0000","last_change":"2024-04-26T21:59:43.059420+0000","last_active":"2024-04-26T21:59:50.313994+0000","last_peered":"2024-04-26T21:59:50.313994+0000","last_clean":"2024-04-26T21:59:50.313994+0000","last_became_active":"2024-04-26T21:59:43.059033+0000","last_became_peered":"2024-04-26T21:59:43.059033+0000","last_unstale":"2024-04-26T21:59:50.313994+0000","last_undegraded":"2024-04-26T21:59:50.313994+0000","last_fullsized":"2024-04-26T21:59:50.313994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:57.633543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661718+0000","last_change":"2024-04-26T21:59:44.045932+0000","last_active":"2024-04-26T21:59:50.661718+0000","last_peered":"2024-04-26T21:59:50.661718+0000","last_clean":"2024-04-26T21:59:50.661718+0000","last_became_active":"2024-04-26T21:59:44.044775+0000","last_became_peered":"2024-04-26T21:59:44.044775+0000","last_unstale":"2024-04-26T21:59:50.661718+0000","last_undegraded":"2024-04-26T21:59:50.661718+0000","last_fullsized":"2024-04-26T21:59:50.661718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:05.719002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323950+0000","last_change":"2024-04-26T21:59:48.151182+0000","last_active":"2024-04-26T21:59:50.323950+0000","last_peered":"2024-04-26T21:59:50.323950+0000","last_clean":"2024-04-26T21:59:50.323950+0000","last_became_active":"2024-04-26T21:59:48.150815+0000","last_became_peered":"2024-04-26T21:59:48.150815+0000","last_unstale":"2024-04-26T21:59:50.323950+0000","last_undegraded":"2024-04-26T21:59:50.323950+0000","last_fullsized":"2024-04-26T21:59:50.323950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.630163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120863+0000","last_change":"2024-04-26T21:59:43.039852+0000","last_active":"2024-04-26T21:59:49.120863+0000","last_peered":"2024-04-26T21:59:49.120863+0000","last_clean":"2024-04-26T21:59:49.120863+0000","last_became_active":"2024-04-26T21:59:43.039499+0000","last_became_peered":"2024-04-26T21:59:43.039499+0000","last_unstale":"2024-04-26T21:59:49.120863+0000","last_undegraded":"2024-04-26T21:59:49.120863+0000","last_fullsized":"2024-04-26T21:59:49.120863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:14.599270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661485+0000","last_change":"2024-04-26T21:59:44.057285+0000","last_active":"2024-04-26T21:59:50.661485+0000","last_peered":"2024-04-26T21:59:50.661485+0000","last_clean":"2024-04-26T21:59:50.661485+0000","last_became_active":"2024-04-26T21:59:44.056947+0000","last_became_peered":"2024-04-26T21:59:44.056947+0000","last_unstale":"2024-04-26T21:59:50.661485+0000","last_undegraded":"2024-04-26T21:59:50.661485+0000","last_fullsized":"2024-04-26T21:59:50.661485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:58.242075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119912+0000","last_change":"2024-04-26T21:59:48.131624+0000","last_active":"2024-04-26T21:59:49.119912+0000","last_peered":"2024-04-26T21:59:49.119912+0000","last_clean":"2024-04-26T21:59:49.119912+0000","last_became_active":"2024-04-26T21:59:48.131193+0000","last_became_peered":"2024-04-26T21:59:48.131193+0000","last_unstale":"2024-04-26T21:59:49.119912+0000","last_undegraded":"2024-04-26T21:59:49.119912+0000","last_fullsized":"2024-04-26T21:59:49.119912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:52.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723555+0000","last_change":"2024-04-26T21:59:43.047723+0000","last_active":"2024-04-26T21:59:50.723555+0000","last_peered":"2024-04-26T21:59:50.723555+0000","last_clean":"2024-04-26T21:59:50.723555+0000","last_became_active":"2024-04-26T21:59:43.045876+0000","last_became_peered":"2024-04-26T21:59:43.045876+0000","last_unstale":"2024-04-26T21:59:50.723555+0000","last_undegraded":"2024-04-26T21:59:50.723555+0000","last_fullsized":"2024-04-26T21:59:50.723555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:37.705490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661733+0000","last_change":"2024-04-26T21:59:44.040723+0000","last_active":"2024-04-26T21:59:50.661733+0000","last_peered":"2024-04-26T21:59:50.661733+0000","last_clean":"2024-04-26T21:59:50.661733+0000","last_became_active":"2024-04-26T21:59:44.040389+0000","last_became_peered":"2024-04-26T21:59:44.040389+0000","last_unstale":"2024-04-26T21:59:50.661733+0000","last_undegraded":"2024-04-26T21:59:50.661733+0000","last_fullsized":"2024-04-26T21:59:50.661733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:05.120695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149410+0000","last_change":"2024-04-26T21:59:44.041027+0000","last_active":"2024-04-26T21:59:51.149410+0000","last_peered":"2024-04-26T21:59:51.149410+0000","last_clean":"2024-04-26T21:59:51.149410+0000","last_became_active":"2024-04-26T21:59:44.040705+0000","last_became_peered":"2024-04-26T21:59:44.040705+0000","last_unstale":"2024-04-26T21:59:51.149410+0000","last_undegraded":"2024-04-26T21:59:51.149410+0000","last_fullsized":"2024-04-26T21:59:51.149410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:11.114850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313739+0000","last_change":"2024-04-26T21:59:44.040916+0000","last_active":"2024-04-26T21:59:50.313739+0000","last_peered":"2024-04-26T21:59:50.313739+0000","last_clean":"2024-04-26T21:59:50.313739+0000","last_became_active":"2024-04-26T21:59:44.040568+0000","last_became_peered":"2024-04-26T21:59:44.040568+0000","last_unstale":"2024-04-26T21:59:50.313739+0000","last_undegraded":"2024-04-26T21:59:50.313739+0000","last_fullsized":"2024-04-26T21:59:50.313739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:04.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313347+0000","last_change":"2024-04-26T21:59:44.046676+0000","last_active":"2024-04-26T21:59:50.313347+0000","last_peered":"2024-04-26T21:59:50.313347+0000","last_clean":"2024-04-26T21:59:50.313347+0000","last_became_active":"2024-04-26T21:59:44.046339+0000","last_became_peered":"2024-04-26T21:59:44.046339+0000","last_unstale":"2024-04-26T21:59:50.313347+0000","last_undegraded":"2024-04-26T21:59:50.313347+0000","last_fullsized":"2024-04-26T21:59:50.313347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:51.964558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324145+0000","last_change":"2024-04-26T21:59:44.064635+0000","last_active":"2024-04-26T21:59:50.324145+0000","last_peered":"2024-04-26T21:59:50.324145+0000","last_clean":"2024-04-26T21:59:50.324145+0000","last_became_active":"2024-04-26T21:59:44.064174+0000","last_became_peered":"2024-04-26T21:59:44.064174+0000","last_unstale":"2024-04-26T21:59:50.324145+0000","last_undegraded":"2024-04-26T21:59:50.324145+0000","last_fullsized":"2024-04-26T21:59:50.324145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:59.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723241+0000","last_change":"2024-04-26T21:59:44.051872+0000","last_active":"2024-04-26T21:59:50.723241+0000","last_peered":"2024-04-26T21:59:50.723241+0000","last_clean":"2024-04-26T21:59:50.723241+0000","last_became_active":"2024-04-26T21:59:44.051539+0000","last_became_peered":"2024-04-26T21:59:44.051539+0000","last_unstale":"2024-04-26T21:59:50.723241+0000","last_undegraded":"2024-04-26T21:59:50.723241+0000","last_fullsized":"2024-04-26T21:59:50.723241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:15.622222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149504+0000","last_change":"2024-04-26T21:59:44.044024+0000","last_active":"2024-04-26T21:59:51.149504+0000","last_peered":"2024-04-26T21:59:51.149504+0000","last_clean":"2024-04-26T21:59:51.149504+0000","last_became_active":"2024-04-26T21:59:44.043718+0000","last_became_peered":"2024-04-26T21:59:44.043718+0000","last_unstale":"2024-04-26T21:59:51.149504+0000","last_undegraded":"2024-04-26T21:59:51.149504+0000","last_fullsized":"2024-04-26T21:59:51.149504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:35.028538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313921+0000","last_change":"2024-04-26T21:59:44.037432+0000","last_active":"2024-04-26T21:59:50.313921+0000","last_peered":"2024-04-26T21:59:50.313921+0000","last_clean":"2024-04-26T21:59:50.313921+0000","last_became_active":"2024-04-26T21:59:44.037086+0000","last_became_peered":"2024-04-26T21:59:44.037086+0000","last_unstale":"2024-04-26T21:59:50.313921+0000","last_undegraded":"2024-04-26T21:59:50.313921+0000","last_fullsized":"2024-04-26T21:59:50.313921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:32.828866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122540+0000","last_change":"2024-04-26T21:59:44.055253+0000","last_active":"2024-04-26T21:59:49.122540+0000","last_peered":"2024-04-26T21:59:49.122540+0000","last_clean":"2024-04-26T21:59:49.122540+0000","last_became_active":"2024-04-26T21:59:44.054329+0000","last_became_peered":"2024-04-26T21:59:44.054329+0000","last_unstale":"2024-04-26T21:59:49.122540+0000","last_undegraded":"2024-04-26T21:59:49.122540+0000","last_fullsized":"2024-04-26T21:59:49.122540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:02.949272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120075+0000","last_change":"2024-04-26T21:59:44.060213+0000","last_active":"2024-04-26T21:59:49.120075+0000","last_peered":"2024-04-26T21:59:49.120075+0000","last_clean":"2024-04-26T21:59:49.120075+0000","last_became_active":"2024-04-26T21:59:44.059987+0000","last_became_peered":"2024-04-26T21:59:44.059987+0000","last_unstale":"2024-04-26T21:59:49.120075+0000","last_undegraded":"2024-04-26T21:59:49.120075+0000","last_fullsized":"2024-04-26T21:59:49.120075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:23.184373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121153+0000","last_change":"2024-04-26T21:59:44.046779+0000","last_active":"2024-04-26T21:59:49.121153+0000","last_peered":"2024-04-26T21:59:49.121153+0000","last_clean":"2024-04-26T21:59:49.121153+0000","last_became_active":"2024-04-26T21:59:44.046304+0000","last_became_peered":"2024-04-26T21:59:44.046304+0000","last_unstale":"2024-04-26T21:59:49.121153+0000","last_undegraded":"2024-04-26T21:59:49.121153+0000","last_fullsized":"2024-04-26T21:59:49.121153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:59.738908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720000+0000","last_change":"2024-04-26T21:59:44.055284+0000","last_active":"2024-04-26T21:59:50.720000+0000","last_peered":"2024-04-26T21:59:50.720000+0000","last_clean":"2024-04-26T21:59:50.720000+0000","last_became_active":"2024-04-26T21:59:44.054897+0000","last_became_peered":"2024-04-26T21:59:44.054897+0000","last_unstale":"2024-04-26T21:59:50.720000+0000","last_undegraded":"2024-04-26T21:59:50.720000+0000","last_fullsized":"2024-04-26T21:59:50.720000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:57.942834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723093+0000","last_change":"2024-04-26T21:59:44.060516+0000","last_active":"2024-04-26T21:59:50.723093+0000","last_peered":"2024-04-26T21:59:50.723093+0000","last_clean":"2024-04-26T21:59:50.723093+0000","last_became_active":"2024-04-26T21:59:44.059564+0000","last_became_peered":"2024-04-26T21:59:44.059564+0000","last_unstale":"2024-04-26T21:59:50.723093+0000","last_undegraded":"2024-04-26T21:59:50.723093+0000","last_fullsized":"2024-04-26T21:59:50.723093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:43.784018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149377+0000","last_change":"2024-04-26T21:59:44.043564+0000","last_active":"2024-04-26T21:59:51.149377+0000","last_peered":"2024-04-26T21:59:51.149377+0000","last_clean":"2024-04-26T21:59:51.149377+0000","last_became_active":"2024-04-26T21:59:44.043254+0000","last_became_peered":"2024-04-26T21:59:44.043254+0000","last_unstale":"2024-04-26T21:59:51.149377+0000","last_undegraded":"2024-04-26T21:59:51.149377+0000","last_fullsized":"2024-04-26T21:59:51.149377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:12.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":11,"num_write":53,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":13920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1076113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037796,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":486409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1080851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462959,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":485551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":481799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":481813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":481731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1605632,"data_stored":1076355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:14.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:14.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:14 smithi012 ceph-mon[28184]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T22:00:14.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/845132530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1579251453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 -- 172.21.15.12:0/174811165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078530 msgr2=0x7efe4c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 --2- 172.21.15.12:0/174811165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078530 0x7efe4c078990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe38009900 tx=0x7efe3802a400 comp rx=0 tx=0).stop 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 -- 172.21.15.12:0/174811165 shutdown_connections 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 --2- 172.21.15.12:0/174811165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c078ed0 0x7efe4c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 --2- 172.21.15.12:0/174811165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078530 0x7efe4c078990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 --2- 172.21.15.12:0/174811165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c0772f0 0x7efe4c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 -- 172.21.15.12:0/174811165 >> 172.21.15.12:0/174811165 conn(0x7efe4c100120 msgr2=0x7efe4c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 -- 172.21.15.12:0/174811165 shutdown_connections 2024-04-26T22:00:14.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.629+0000 7efe54200640 1 -- 172.21.15.12:0/174811165 wait complete. 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 Processor -- start 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 -- start start 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c0772f0 0x7efe4c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 0x7efe4c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c114a70 con 0x7efe4c078530 2024-04-26T22:00:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1148f0 con 0x7efe4c078ed0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.630+0000 7efe54200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c114bf0 con 0x7efe4c0772f0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe50e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe50e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43688/0 (socket says 172.21.15.12:43688) 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe50e00640 1 -- 172.21.15.12:0/733270261 learned_addr learned my addr 172.21.15.12:0/733270261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 0x7efe4c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe51800640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c0772f0 0x7efe4c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 -- 172.21.15.12:0/733270261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c0772f0 msgr2=0x7efe4c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c0772f0 0x7efe4c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 -- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 msgr2=0x7efe4c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe52200640 1 -- 172.21.15.12:0/733270261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a6dc0 con 0x7efe4c078ed0 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.631+0000 7efe50e00640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:14.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe52200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 0x7efe4c1a6750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e940 tx=0x7efe4400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004d20 con 0x7efe4c078ed0 2024-04-26T22:00:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44005020 con 0x7efe4c078ed0 2024-04-26T22:00:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a70b0 con 0x7efe4c078ed0 2024-04-26T22:00:14.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440116d0 con 0x7efe4c078ed0 2024-04-26T22:00:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.632+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a7580 con 0x7efe4c078ed0 2024-04-26T22:00:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.633+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c1114e0 con 0x7efe4c078ed0 2024-04-26T22:00:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.635+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efe44005460 con 0x7efe4c078ed0 2024-04-26T22:00:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.637+0000 7efe4a000640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 0x7efe340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.637+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efe440d08c0 con 0x7efe4c078ed0 2024-04-26T22:00:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.637+0000 7efe51800640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 0x7efe340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.638+0000 7efe51800640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 0x7efe340a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe4000bbe0 tx=0x7efe4000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:14.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:14 smithi154 ceph-mon[32578]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T22:00:14.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/845132530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1579251453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.641+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe44016030 con 0x7efe4c078ed0 2024-04-26T22:00:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:14 smithi077 ceph-mon[32280]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 597 B/s wr, 1 op/s 2024-04-26T22:00:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/845132530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1579251453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:14.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.842+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe4c062d70 con 0x7efe340a3c40 2024-04-26T22:00:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.859+0000 7efe4a000640 1 -- 172.21.15.12:0/733270261 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340218 (secure 0 0 0) 0x7efe4c062d70 con 0x7efe340a3c40 2024-04-26T22:00:14.865 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:14.867 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T22:00:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.865+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 msgr2=0x7efe340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.865+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 0x7efe340a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe4000bbe0 tx=0x7efe4000b390 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.865+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 msgr2=0x7efe4c1a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.865+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 0x7efe4c1a6750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e940 tx=0x7efe4400ee00 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 shutdown_connections 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efe340a3c40 0x7efe340a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe4c078ed0 0x7efe4c1a6750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c078530 0x7efe4c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 --2- 172.21.15.12:0/733270261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4c0772f0 0x7efe4c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 >> 172.21.15.12:0/733270261 conn(0x7efe4c100120 msgr2=0x7efe4c101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 shutdown_connections 2024-04-26T22:00:14.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:14.866+0000 7efe54200640 1 -- 172.21.15.12:0/733270261 wait complete. 2024-04-26T22:00:14.888 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-04-26T22:00:13.130613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595125,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":145,"num_read_kb":130,"num_write":235,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379344,"kb_used_data":13524,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562480,"statfs":{"total":1151940427776,"available":1151551979520,"internally_reserved":0,"allocated":13848576,"data_stored":7590108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1898,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":11,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005698"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149711+0000","last_change":"2024-04-26T21:59:44.048850+0000","last_active":"2024-04-26T21:59:51.149711+0000","last_peered":"2024-04-26T21:59:51.149711+0000","last_clean":"2024-04-26T21:59:51.149711+0000","last_became_active":"2024-04-26T21:59:44.048521+0000","last_became_peered":"2024-04-26T21:59:44.048521+0000","last_unstale":"2024-04-26T21:59:51.149711+0000","last_undegraded":"2024-04-26T21:59:51.149711+0000","last_fullsized":"2024-04-26T21:59:51.149711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:09.031610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324469+0000","last_change":"2024-04-26T21:59:44.048900+0000","last_active":"2024-04-26T21:59:50.324469+0000","last_peered":"2024-04-26T21:59:50.324469+0000","last_clean":"2024-04-26T21:59:50.324469+0000","last_became_active":"2024-04-26T21:59:44.048475+0000","last_became_peered":"2024-04-26T21:59:44.048475+0000","last_unstale":"2024-04-26T21:59:50.324469+0000","last_undegraded":"2024-04-26T21:59:50.324469+0000","last_fullsized":"2024-04-26T21:59:50.324469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:01.724256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324301+0000","last_change":"2024-04-26T21:59:44.045875+0000","last_active":"2024-04-26T21:59:50.324301+0000","last_peered":"2024-04-26T21:59:50.324301+0000","last_clean":"2024-04-26T21:59:50.324301+0000","last_became_active":"2024-04-26T21:59:44.045413+0000","last_became_peered":"2024-04-26T21:59:44.045413+0000","last_unstale":"2024-04-26T21:59:50.324301+0000","last_undegraded":"2024-04-26T21:59:50.324301+0000","last_fullsized":"2024-04-26T21:59:50.324301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:19.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118365+0000","last_change":"2024-04-26T21:59:44.042513+0000","last_active":"2024-04-26T21:59:49.118365+0000","last_peered":"2024-04-26T21:59:49.118365+0000","last_clean":"2024-04-26T21:59:49.118365+0000","last_became_active":"2024-04-26T21:59:44.042156+0000","last_became_peered":"2024-04-26T21:59:44.042156+0000","last_unstale":"2024-04-26T21:59:49.118365+0000","last_undegraded":"2024-04-26T21:59:49.118365+0000","last_fullsized":"2024-04-26T21:59:49.118365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:08.494602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719554+0000","last_change":"2024-04-26T21:59:44.063199+0000","last_active":"2024-04-26T21:59:50.719554+0000","last_peered":"2024-04-26T21:59:50.719554+0000","last_clean":"2024-04-26T21:59:50.719554+0000","last_became_active":"2024-04-26T21:59:44.062845+0000","last_became_peered":"2024-04-26T21:59:44.062845+0000","last_unstale":"2024-04-26T21:59:50.719554+0000","last_undegraded":"2024-04-26T21:59:50.719554+0000","last_fullsized":"2024-04-26T21:59:50.719554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:36.247652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388977+0000","last_change":"2024-04-26T21:59:44.058069+0000","last_active":"2024-04-26T21:59:50.388977+0000","last_peered":"2024-04-26T21:59:50.388977+0000","last_clean":"2024-04-26T21:59:50.388977+0000","last_became_active":"2024-04-26T21:59:44.057726+0000","last_became_peered":"2024-04-26T21:59:44.057726+0000","last_unstale":"2024-04-26T21:59:50.388977+0000","last_undegraded":"2024-04-26T21:59:50.388977+0000","last_fullsized":"2024-04-26T21:59:50.388977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:37.887034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661192+0000","last_change":"2024-04-26T21:59:44.046887+0000","last_active":"2024-04-26T21:59:50.661192+0000","last_peered":"2024-04-26T21:59:50.661192+0000","last_clean":"2024-04-26T21:59:50.661192+0000","last_became_active":"2024-04-26T21:59:44.046507+0000","last_became_peered":"2024-04-26T21:59:44.046507+0000","last_unstale":"2024-04-26T21:59:50.661192+0000","last_undegraded":"2024-04-26T21:59:50.661192+0000","last_fullsized":"2024-04-26T21:59:50.661192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:03.312480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119800+0000","last_change":"2024-04-26T21:59:44.060437+0000","last_active":"2024-04-26T21:59:49.119800+0000","last_peered":"2024-04-26T21:59:49.119800+0000","last_clean":"2024-04-26T21:59:49.119800+0000","last_became_active":"2024-04-26T21:59:44.055341+0000","last_became_peered":"2024-04-26T21:59:44.055341+0000","last_unstale":"2024-04-26T21:59:49.119800+0000","last_undegraded":"2024-04-26T21:59:49.119800+0000","last_fullsized":"2024-04-26T21:59:49.119800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:22.452619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720226+0000","last_change":"2024-04-26T21:59:44.043537+0000","last_active":"2024-04-26T21:59:50.720226+0000","last_peered":"2024-04-26T21:59:50.720226+0000","last_clean":"2024-04-26T21:59:50.720226+0000","last_became_active":"2024-04-26T21:59:44.042227+0000","last_became_peered":"2024-04-26T21:59:44.042227+0000","last_unstale":"2024-04-26T21:59:50.720226+0000","last_undegraded":"2024-04-26T21:59:50.720226+0000","last_fullsized":"2024-04-26T21:59:50.720226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.663380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389294+0000","last_change":"2024-04-26T21:59:44.039338+0000","last_active":"2024-04-26T21:59:50.389294+0000","last_peered":"2024-04-26T21:59:50.389294+0000","last_clean":"2024-04-26T21:59:50.389294+0000","last_became_active":"2024-04-26T21:59:44.039000+0000","last_became_peered":"2024-04-26T21:59:44.039000+0000","last_unstale":"2024-04-26T21:59:50.389294+0000","last_undegraded":"2024-04-26T21:59:50.389294+0000","last_fullsized":"2024-04-26T21:59:50.389294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:20.829507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661671+0000","last_change":"2024-04-26T21:59:44.046272+0000","last_active":"2024-04-26T21:59:50.661671+0000","last_peered":"2024-04-26T21:59:50.661671+0000","last_clean":"2024-04-26T21:59:50.661671+0000","last_became_active":"2024-04-26T21:59:44.045235+0000","last_became_peered":"2024-04-26T21:59:44.045235+0000","last_unstale":"2024-04-26T21:59:50.661671+0000","last_undegraded":"2024-04-26T21:59:50.661671+0000","last_fullsized":"2024-04-26T21:59:50.661671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118216+0000","last_change":"2024-04-26T21:59:44.044698+0000","last_active":"2024-04-26T21:59:49.118216+0000","last_peered":"2024-04-26T21:59:49.118216+0000","last_clean":"2024-04-26T21:59:49.118216+0000","last_became_active":"2024-04-26T21:59:44.043394+0000","last_became_peered":"2024-04-26T21:59:44.043394+0000","last_unstale":"2024-04-26T21:59:49.118216+0000","last_undegraded":"2024-04-26T21:59:49.118216+0000","last_fullsized":"2024-04-26T21:59:49.118216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:55.254581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723599+0000","last_change":"2024-04-26T21:59:44.060896+0000","last_active":"2024-04-26T21:59:50.723599+0000","last_peered":"2024-04-26T21:59:50.723599+0000","last_clean":"2024-04-26T21:59:50.723599+0000","last_became_active":"2024-04-26T21:59:44.060010+0000","last_became_peered":"2024-04-26T21:59:44.060010+0000","last_unstale":"2024-04-26T21:59:50.723599+0000","last_undegraded":"2024-04-26T21:59:50.723599+0000","last_fullsized":"2024-04-26T21:59:50.723599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:46.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323909+0000","last_change":"2024-04-26T21:59:44.050761+0000","last_active":"2024-04-26T21:59:50.323909+0000","last_peered":"2024-04-26T21:59:50.323909+0000","last_clean":"2024-04-26T21:59:50.323909+0000","last_became_active":"2024-04-26T21:59:44.050229+0000","last_became_peered":"2024-04-26T21:59:44.050229+0000","last_unstale":"2024-04-26T21:59:50.323909+0000","last_undegraded":"2024-04-26T21:59:50.323909+0000","last_fullsized":"2024-04-26T21:59:50.323909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:46.114433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323698+0000","last_change":"2024-04-26T21:59:44.056593+0000","last_active":"2024-04-26T21:59:50.323698+0000","last_peered":"2024-04-26T21:59:50.323698+0000","last_clean":"2024-04-26T21:59:50.323698+0000","last_became_active":"2024-04-26T21:59:44.056111+0000","last_became_peered":"2024-04-26T21:59:44.056111+0000","last_unstale":"2024-04-26T21:59:50.323698+0000","last_undegraded":"2024-04-26T21:59:50.323698+0000","last_fullsized":"2024-04-26T21:59:50.323698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:34.118566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719667+0000","last_change":"2024-04-26T21:59:44.049616+0000","last_active":"2024-04-26T21:59:50.719667+0000","last_peered":"2024-04-26T21:59:50.719667+0000","last_clean":"2024-04-26T21:59:50.719667+0000","last_became_active":"2024-04-26T21:59:44.049259+0000","last_became_peered":"2024-04-26T21:59:44.049259+0000","last_unstale":"2024-04-26T21:59:50.719667+0000","last_undegraded":"2024-04-26T21:59:50.719667+0000","last_fullsized":"2024-04-26T21:59:50.719667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:25.264219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149586+0000","last_change":"2024-04-26T21:59:44.048657+0000","last_active":"2024-04-26T21:59:51.149586+0000","last_peered":"2024-04-26T21:59:51.149586+0000","last_clean":"2024-04-26T21:59:51.149586+0000","last_became_active":"2024-04-26T21:59:44.048209+0000","last_became_peered":"2024-04-26T21:59:44.048209+0000","last_unstale":"2024-04-26T21:59:51.149586+0000","last_undegraded":"2024-04-26T21:59:51.149586+0000","last_fullsized":"2024-04-26T21:59:51.149586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:01.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459579+0000","last_change":"2024-04-26T21:59:44.058215+0000","last_active":"2024-04-26T21:59:49.459579+0000","last_peered":"2024-04-26T21:59:49.459579+0000","last_clean":"2024-04-26T21:59:49.459579+0000","last_became_active":"2024-04-26T21:59:44.057422+0000","last_became_peered":"2024-04-26T21:59:44.057422+0000","last_unstale":"2024-04-26T21:59:49.459579+0000","last_undegraded":"2024-04-26T21:59:49.459579+0000","last_fullsized":"2024-04-26T21:59:49.459579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:17.215168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313975+0000","last_change":"2024-04-26T21:59:44.054840+0000","last_active":"2024-04-26T21:59:50.313975+0000","last_peered":"2024-04-26T21:59:50.313975+0000","last_clean":"2024-04-26T21:59:50.313975+0000","last_became_active":"2024-04-26T21:59:44.054629+0000","last_became_peered":"2024-04-26T21:59:44.054629+0000","last_unstale":"2024-04-26T21:59:50.313975+0000","last_undegraded":"2024-04-26T21:59:50.313975+0000","last_fullsized":"2024-04-26T21:59:50.313975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:37.204066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313510+0000","last_change":"2024-04-26T21:59:48.145585+0000","last_active":"2024-04-26T21:59:50.313510+0000","last_peered":"2024-04-26T21:59:50.313510+0000","last_clean":"2024-04-26T21:59:50.313510+0000","last_became_active":"2024-04-26T21:59:48.145283+0000","last_became_peered":"2024-04-26T21:59:48.145283+0000","last_unstale":"2024-04-26T21:59:50.313510+0000","last_undegraded":"2024-04-26T21:59:50.313510+0000","last_fullsized":"2024-04-26T21:59:50.313510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:53.096333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.460095+0000","last_change":"2024-04-26T21:59:43.059127+0000","last_active":"2024-04-26T21:59:49.460095+0000","last_peered":"2024-04-26T21:59:49.460095+0000","last_clean":"2024-04-26T21:59:49.460095+0000","last_became_active":"2024-04-26T21:59:43.057996+0000","last_became_peered":"2024-04-26T21:59:43.057996+0000","last_unstale":"2024-04-26T21:59:49.460095+0000","last_undegraded":"2024-04-26T21:59:49.460095+0000","last_fullsized":"2024-04-26T21:59:49.460095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:06.086437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121706+0000","last_change":"2024-04-26T21:59:44.060192+0000","last_active":"2024-04-26T21:59:49.121706+0000","last_peered":"2024-04-26T21:59:49.121706+0000","last_clean":"2024-04-26T21:59:49.121706+0000","last_became_active":"2024-04-26T21:59:44.054969+0000","last_became_peered":"2024-04-26T21:59:44.054969+0000","last_unstale":"2024-04-26T21:59:49.121706+0000","last_undegraded":"2024-04-26T21:59:49.121706+0000","last_fullsized":"2024-04-26T21:59:49.121706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:54.670629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117835+0000","last_change":"2024-04-26T21:59:48.144947+0000","last_active":"2024-04-26T21:59:49.117835+0000","last_peered":"2024-04-26T21:59:49.117835+0000","last_clean":"2024-04-26T21:59:49.117835+0000","last_became_active":"2024-04-26T21:59:48.144547+0000","last_became_peered":"2024-04-26T21:59:48.144547+0000","last_unstale":"2024-04-26T21:59:49.117835+0000","last_undegraded":"2024-04-26T21:59:49.117835+0000","last_fullsized":"2024-04-26T21:59:49.117835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.492136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122109+0000","last_change":"2024-04-26T21:59:43.071253+0000","last_active":"2024-04-26T21:59:49.122109+0000","last_peered":"2024-04-26T21:59:49.122109+0000","last_clean":"2024-04-26T21:59:49.122109+0000","last_became_active":"2024-04-26T21:59:43.070926+0000","last_became_peered":"2024-04-26T21:59:43.070926+0000","last_unstale":"2024-04-26T21:59:49.122109+0000","last_undegraded":"2024-04-26T21:59:49.122109+0000","last_fullsized":"2024-04-26T21:59:49.122109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:13.280936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324550+0000","last_change":"2024-04-26T21:59:44.051951+0000","last_active":"2024-04-26T21:59:50.324550+0000","last_peered":"2024-04-26T21:59:50.324550+0000","last_clean":"2024-04-26T21:59:50.324550+0000","last_became_active":"2024-04-26T21:59:44.051206+0000","last_became_peered":"2024-04-26T21:59:44.051206+0000","last_unstale":"2024-04-26T21:59:50.324550+0000","last_undegraded":"2024-04-26T21:59:50.324550+0000","last_fullsized":"2024-04-26T21:59:50.324550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:05.034106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117969+0000","last_change":"2024-04-26T21:59:48.150369+0000","last_active":"2024-04-26T21:59:49.117969+0000","last_peered":"2024-04-26T21:59:49.117969+0000","last_clean":"2024-04-26T21:59:49.117969+0000","last_became_active":"2024-04-26T21:59:48.149632+0000","last_became_peered":"2024-04-26T21:59:48.149632+0000","last_unstale":"2024-04-26T21:59:49.117969+0000","last_undegraded":"2024-04-26T21:59:49.117969+0000","last_fullsized":"2024-04-26T21:59:49.117969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:05.263939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720074+0000","last_change":"2024-04-26T21:59:43.038322+0000","last_active":"2024-04-26T21:59:50.720074+0000","last_peered":"2024-04-26T21:59:50.720074+0000","last_clean":"2024-04-26T21:59:50.720074+0000","last_became_active":"2024-04-26T21:59:43.037968+0000","last_became_peered":"2024-04-26T21:59:43.037968+0000","last_unstale":"2024-04-26T21:59:50.720074+0000","last_undegraded":"2024-04-26T21:59:50.720074+0000","last_fullsized":"2024-04-26T21:59:50.720074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:21.265367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720035+0000","last_change":"2024-04-26T21:59:44.054970+0000","last_active":"2024-04-26T21:59:50.720035+0000","last_peered":"2024-04-26T21:59:50.720035+0000","last_clean":"2024-04-26T21:59:50.720035+0000","last_became_active":"2024-04-26T21:59:44.054599+0000","last_became_peered":"2024-04-26T21:59:44.054599+0000","last_unstale":"2024-04-26T21:59:50.720035+0000","last_undegraded":"2024-04-26T21:59:50.720035+0000","last_fullsized":"2024-04-26T21:59:50.720035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:22.048544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117821+0000","last_change":"2024-04-26T21:59:48.136704+0000","last_active":"2024-04-26T21:59:49.117821+0000","last_peered":"2024-04-26T21:59:49.117821+0000","last_clean":"2024-04-26T21:59:49.117821+0000","last_became_active":"2024-04-26T21:59:48.136276+0000","last_became_peered":"2024-04-26T21:59:48.136276+0000","last_unstale":"2024-04-26T21:59:49.117821+0000","last_undegraded":"2024-04-26T21:59:49.117821+0000","last_fullsized":"2024-04-26T21:59:49.117821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:05.608497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719799+0000","last_change":"2024-04-26T21:59:43.031556+0000","last_active":"2024-04-26T21:59:50.719799+0000","last_peered":"2024-04-26T21:59:50.719799+0000","last_clean":"2024-04-26T21:59:50.719799+0000","last_became_active":"2024-04-26T21:59:43.031216+0000","last_became_peered":"2024-04-26T21:59:43.031216+0000","last_unstale":"2024-04-26T21:59:50.719799+0000","last_undegraded":"2024-04-26T21:59:50.719799+0000","last_fullsized":"2024-04-26T21:59:50.719799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:51.675630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120287+0000","last_change":"2024-04-26T21:59:44.048291+0000","last_active":"2024-04-26T21:59:49.120287+0000","last_peered":"2024-04-26T21:59:49.120287+0000","last_clean":"2024-04-26T21:59:49.120287+0000","last_became_active":"2024-04-26T21:59:44.047983+0000","last_became_peered":"2024-04-26T21:59:44.047983+0000","last_unstale":"2024-04-26T21:59:49.120287+0000","last_undegraded":"2024-04-26T21:59:49.120287+0000","last_fullsized":"2024-04-26T21:59:49.120287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:17.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313826+0000","last_change":"2024-04-26T21:59:48.129287+0000","last_active":"2024-04-26T21:59:50.313826+0000","last_peered":"2024-04-26T21:59:50.313826+0000","last_clean":"2024-04-26T21:59:50.313826+0000","last_became_active":"2024-04-26T21:59:48.128789+0000","last_became_peered":"2024-04-26T21:59:48.128789+0000","last_unstale":"2024-04-26T21:59:50.313826+0000","last_undegraded":"2024-04-26T21:59:50.313826+0000","last_fullsized":"2024-04-26T21:59:50.313826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:46.495215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120397+0000","last_change":"2024-04-26T21:59:43.034634+0000","last_active":"2024-04-26T21:59:49.120397+0000","last_peered":"2024-04-26T21:59:49.120397+0000","last_clean":"2024-04-26T21:59:49.120397+0000","last_became_active":"2024-04-26T21:59:43.034085+0000","last_became_peered":"2024-04-26T21:59:43.034085+0000","last_unstale":"2024-04-26T21:59:49.120397+0000","last_undegraded":"2024-04-26T21:59:49.120397+0000","last_fullsized":"2024-04-26T21:59:49.120397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:16:35.415892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719745+0000","last_change":"2024-04-26T21:59:44.058077+0000","last_active":"2024-04-26T21:59:50.719745+0000","last_peered":"2024-04-26T21:59:50.719745+0000","last_clean":"2024-04-26T21:59:50.719745+0000","last_became_active":"2024-04-26T21:59:44.057720+0000","last_became_peered":"2024-04-26T21:59:44.057720+0000","last_unstale":"2024-04-26T21:59:50.719745+0000","last_undegraded":"2024-04-26T21:59:50.719745+0000","last_fullsized":"2024-04-26T21:59:50.719745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:56.889733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313754+0000","last_change":"2024-04-26T21:59:48.149256+0000","last_active":"2024-04-26T21:59:50.313754+0000","last_peered":"2024-04-26T21:59:50.313754+0000","last_clean":"2024-04-26T21:59:50.313754+0000","last_became_active":"2024-04-26T21:59:48.148832+0000","last_became_peered":"2024-04-26T21:59:48.148832+0000","last_unstale":"2024-04-26T21:59:50.313754+0000","last_undegraded":"2024-04-26T21:59:50.313754+0000","last_fullsized":"2024-04-26T21:59:50.313754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:46.120634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388794+0000","last_change":"2024-04-26T21:59:43.050527+0000","last_active":"2024-04-26T21:59:50.388794+0000","last_peered":"2024-04-26T21:59:50.388794+0000","last_clean":"2024-04-26T21:59:50.388794+0000","last_became_active":"2024-04-26T21:59:43.050168+0000","last_became_peered":"2024-04-26T21:59:43.050168+0000","last_unstale":"2024-04-26T21:59:50.388794+0000","last_undegraded":"2024-04-26T21:59:50.388794+0000","last_fullsized":"2024-04-26T21:59:50.388794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:45.484392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120735+0000","last_change":"2024-04-26T21:59:44.063479+0000","last_active":"2024-04-26T21:59:49.120735+0000","last_peered":"2024-04-26T21:59:49.120735+0000","last_clean":"2024-04-26T21:59:49.120735+0000","last_became_active":"2024-04-26T21:59:44.063205+0000","last_became_peered":"2024-04-26T21:59:44.063205+0000","last_unstale":"2024-04-26T21:59:49.120735+0000","last_undegraded":"2024-04-26T21:59:49.120735+0000","last_fullsized":"2024-04-26T21:59:49.120735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:07.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149596+0000","last_change":"2024-04-26T21:59:48.155383+0000","last_active":"2024-04-26T21:59:51.149596+0000","last_peered":"2024-04-26T21:59:51.149596+0000","last_clean":"2024-04-26T21:59:51.149596+0000","last_became_active":"2024-04-26T21:59:48.154948+0000","last_became_peered":"2024-04-26T21:59:48.154948+0000","last_unstale":"2024-04-26T21:59:51.149596+0000","last_undegraded":"2024-04-26T21:59:51.149596+0000","last_fullsized":"2024-04-26T21:59:51.149596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:25.874999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722149+0000","last_change":"2024-04-26T21:59:43.061828+0000","last_active":"2024-04-26T21:59:50.722149+0000","last_peered":"2024-04-26T21:59:50.722149+0000","last_clean":"2024-04-26T21:59:50.722149+0000","last_became_active":"2024-04-26T21:59:43.061440+0000","last_became_peered":"2024-04-26T21:59:43.061440+0000","last_unstale":"2024-04-26T21:59:50.722149+0000","last_undegraded":"2024-04-26T21:59:50.722149+0000","last_fullsized":"2024-04-26T21:59:50.722149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:17.964410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120513+0000","last_change":"2024-04-26T21:59:44.056947+0000","last_active":"2024-04-26T21:59:49.120513+0000","last_peered":"2024-04-26T21:59:49.120513+0000","last_clean":"2024-04-26T21:59:49.120513+0000","last_became_active":"2024-04-26T21:59:44.056537+0000","last_became_peered":"2024-04-26T21:59:44.056537+0000","last_unstale":"2024-04-26T21:59:49.120513+0000","last_undegraded":"2024-04-26T21:59:49.120513+0000","last_fullsized":"2024-04-26T21:59:49.120513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:19.601355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324467+0000","last_change":"2024-04-26T21:59:48.141476+0000","last_active":"2024-04-26T21:59:50.324467+0000","last_peered":"2024-04-26T21:59:50.324467+0000","last_clean":"2024-04-26T21:59:50.324467+0000","last_became_active":"2024-04-26T21:59:48.141028+0000","last_became_peered":"2024-04-26T21:59:48.141028+0000","last_unstale":"2024-04-26T21:59:50.324467+0000","last_undegraded":"2024-04-26T21:59:50.324467+0000","last_fullsized":"2024-04-26T21:59:50.324467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:36.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119924+0000","last_change":"2024-04-26T21:59:43.035587+0000","last_active":"2024-04-26T21:59:49.119924+0000","last_peered":"2024-04-26T21:59:49.119924+0000","last_clean":"2024-04-26T21:59:49.119924+0000","last_became_active":"2024-04-26T21:59:43.035102+0000","last_became_peered":"2024-04-26T21:59:43.035102+0000","last_unstale":"2024-04-26T21:59:49.119924+0000","last_undegraded":"2024-04-26T21:59:49.119924+0000","last_fullsized":"2024-04-26T21:59:49.119924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:52.817321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458214+0000","last_change":"2024-04-26T21:59:44.062002+0000","last_active":"2024-04-26T21:59:49.458214+0000","last_peered":"2024-04-26T21:59:49.458214+0000","last_clean":"2024-04-26T21:59:49.458214+0000","last_became_active":"2024-04-26T21:59:44.058419+0000","last_became_peered":"2024-04-26T21:59:44.058419+0000","last_unstale":"2024-04-26T21:59:49.458214+0000","last_undegraded":"2024-04-26T21:59:49.458214+0000","last_fullsized":"2024-04-26T21:59:49.458214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:48.698093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120447+0000","last_change":"2024-04-26T21:59:48.134540+0000","last_active":"2024-04-26T21:59:49.120447+0000","last_peered":"2024-04-26T21:59:49.120447+0000","last_clean":"2024-04-26T21:59:49.120447+0000","last_became_active":"2024-04-26T21:59:48.133149+0000","last_became_peered":"2024-04-26T21:59:48.133149+0000","last_unstale":"2024-04-26T21:59:49.120447+0000","last_undegraded":"2024-04-26T21:59:49.120447+0000","last_fullsized":"2024-04-26T21:59:49.120447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:18.861332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389201+0000","last_change":"2024-04-26T21:59:43.053360+0000","last_active":"2024-04-26T21:59:50.389201+0000","last_peered":"2024-04-26T21:59:50.389201+0000","last_clean":"2024-04-26T21:59:50.389201+0000","last_became_active":"2024-04-26T21:59:43.053113+0000","last_became_peered":"2024-04-26T21:59:43.053113+0000","last_unstale":"2024-04-26T21:59:50.389201+0000","last_undegraded":"2024-04-26T21:59:50.389201+0000","last_fullsized":"2024-04-26T21:59:50.389201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:38.464573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720168+0000","last_change":"2024-04-26T21:59:44.060319+0000","last_active":"2024-04-26T21:59:50.720168+0000","last_peered":"2024-04-26T21:59:50.720168+0000","last_clean":"2024-04-26T21:59:50.720168+0000","last_became_active":"2024-04-26T21:59:44.059947+0000","last_became_peered":"2024-04-26T21:59:44.059947+0000","last_unstale":"2024-04-26T21:59:50.720168+0000","last_undegraded":"2024-04-26T21:59:50.720168+0000","last_fullsized":"2024-04-26T21:59:50.720168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:01.851669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458423+0000","last_change":"2024-04-26T21:59:48.148358+0000","last_active":"2024-04-26T21:59:49.458423+0000","last_peered":"2024-04-26T21:59:49.458423+0000","last_clean":"2024-04-26T21:59:49.458423+0000","last_became_active":"2024-04-26T21:59:48.143520+0000","last_became_peered":"2024-04-26T21:59:48.143520+0000","last_unstale":"2024-04-26T21:59:49.458423+0000","last_undegraded":"2024-04-26T21:59:49.458423+0000","last_fullsized":"2024-04-26T21:59:49.458423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:23.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661626+0000","last_change":"2024-04-26T21:59:43.037153+0000","last_active":"2024-04-26T21:59:50.661626+0000","last_peered":"2024-04-26T21:59:50.661626+0000","last_clean":"2024-04-26T21:59:50.661626+0000","last_became_active":"2024-04-26T21:59:43.036816+0000","last_became_peered":"2024-04-26T21:59:43.036816+0000","last_unstale":"2024-04-26T21:59:50.661626+0000","last_undegraded":"2024-04-26T21:59:50.661626+0000","last_fullsized":"2024-04-26T21:59:50.661626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:44.655304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324672+0000","last_change":"2024-04-26T21:59:44.047947+0000","last_active":"2024-04-26T21:59:50.324672+0000","last_peered":"2024-04-26T21:59:50.324672+0000","last_clean":"2024-04-26T21:59:50.324672+0000","last_became_active":"2024-04-26T21:59:44.047586+0000","last_became_peered":"2024-04-26T21:59:44.047586+0000","last_unstale":"2024-04-26T21:59:50.324672+0000","last_undegraded":"2024-04-26T21:59:50.324672+0000","last_fullsized":"2024-04-26T21:59:50.324672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:49.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118267+0000","last_change":"2024-04-26T21:59:48.150787+0000","last_active":"2024-04-26T21:59:49.118267+0000","last_peered":"2024-04-26T21:59:49.118267+0000","last_clean":"2024-04-26T21:59:49.118267+0000","last_became_active":"2024-04-26T21:59:48.150004+0000","last_became_peered":"2024-04-26T21:59:48.150004+0000","last_unstale":"2024-04-26T21:59:49.118267+0000","last_undegraded":"2024-04-26T21:59:49.118267+0000","last_fullsized":"2024-04-26T21:59:49.118267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:19.657837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120367+0000","last_change":"2024-04-26T21:59:43.050931+0000","last_active":"2024-04-26T21:59:49.120367+0000","last_peered":"2024-04-26T21:59:49.120367+0000","last_clean":"2024-04-26T21:59:49.120367+0000","last_became_active":"2024-04-26T21:59:43.050508+0000","last_became_peered":"2024-04-26T21:59:43.050508+0000","last_unstale":"2024-04-26T21:59:49.120367+0000","last_undegraded":"2024-04-26T21:59:49.120367+0000","last_fullsized":"2024-04-26T21:59:49.120367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:08.378649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720132+0000","last_change":"2024-04-26T21:59:44.039078+0000","last_active":"2024-04-26T21:59:50.720132+0000","last_peered":"2024-04-26T21:59:50.720132+0000","last_clean":"2024-04-26T21:59:50.720132+0000","last_became_active":"2024-04-26T21:59:44.038734+0000","last_became_peered":"2024-04-26T21:59:44.038734+0000","last_unstale":"2024-04-26T21:59:50.720132+0000","last_undegraded":"2024-04-26T21:59:50.720132+0000","last_fullsized":"2024-04-26T21:59:50.720132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:07.551134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.458575+0000","last_change":"2024-04-26T21:59:48.150464+0000","last_active":"2024-04-26T21:59:49.458575+0000","last_peered":"2024-04-26T21:59:49.458575+0000","last_clean":"2024-04-26T21:59:49.458575+0000","last_became_active":"2024-04-26T21:59:48.150095+0000","last_became_peered":"2024-04-26T21:59:48.150095+0000","last_unstale":"2024-04-26T21:59:49.458575+0000","last_undegraded":"2024-04-26T21:59:49.458575+0000","last_fullsized":"2024-04-26T21:59:49.458575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:53.152840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:00:09.250045+0000","last_change":"2024-04-26T21:59:43.038751+0000","last_active":"2024-04-26T22:00:09.250045+0000","last_peered":"2024-04-26T22:00:09.250045+0000","last_clean":"2024-04-26T22:00:09.250045+0000","last_became_active":"2024-04-26T21:59:43.038337+0000","last_became_peered":"2024-04-26T21:59:43.038337+0000","last_unstale":"2024-04-26T22:00:09.250045+0000","last_undegraded":"2024-04-26T22:00:09.250045+0000","last_fullsized":"2024-04-26T22:00:09.250045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.830357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719914+0000","last_change":"2024-04-26T21:59:44.040170+0000","last_active":"2024-04-26T21:59:50.719914+0000","last_peered":"2024-04-26T21:59:50.719914+0000","last_clean":"2024-04-26T21:59:50.719914+0000","last_became_active":"2024-04-26T21:59:44.039713+0000","last_became_peered":"2024-04-26T21:59:44.039713+0000","last_unstale":"2024-04-26T21:59:50.719914+0000","last_undegraded":"2024-04-26T21:59:50.719914+0000","last_fullsized":"2024-04-26T21:59:50.719914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:49.036612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121087+0000","last_change":"2024-04-26T21:59:48.140066+0000","last_active":"2024-04-26T21:59:49.121087+0000","last_peered":"2024-04-26T21:59:49.121087+0000","last_clean":"2024-04-26T21:59:49.121087+0000","last_became_active":"2024-04-26T21:59:48.139748+0000","last_became_peered":"2024-04-26T21:59:48.139748+0000","last_unstale":"2024-04-26T21:59:49.121087+0000","last_undegraded":"2024-04-26T21:59:49.121087+0000","last_fullsized":"2024-04-26T21:59:49.121087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:12.280574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313643+0000","last_change":"2024-04-26T21:59:43.039994+0000","last_active":"2024-04-26T21:59:50.313643+0000","last_peered":"2024-04-26T21:59:50.313643+0000","last_clean":"2024-04-26T21:59:50.313643+0000","last_became_active":"2024-04-26T21:59:43.039671+0000","last_became_peered":"2024-04-26T21:59:43.039671+0000","last_unstale":"2024-04-26T21:59:50.313643+0000","last_undegraded":"2024-04-26T21:59:50.313643+0000","last_fullsized":"2024-04-26T21:59:50.313643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:59.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722856+0000","last_change":"2024-04-26T21:59:44.065515+0000","last_active":"2024-04-26T21:59:50.722856+0000","last_peered":"2024-04-26T21:59:50.722856+0000","last_clean":"2024-04-26T21:59:50.722856+0000","last_became_active":"2024-04-26T21:59:44.065120+0000","last_became_peered":"2024-04-26T21:59:44.065120+0000","last_unstale":"2024-04-26T21:59:50.722856+0000","last_undegraded":"2024-04-26T21:59:50.722856+0000","last_fullsized":"2024-04-26T21:59:50.722856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:09.709198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149237+0000","last_change":"2024-04-26T21:59:48.149372+0000","last_active":"2024-04-26T21:59:51.149237+0000","last_peered":"2024-04-26T21:59:51.149237+0000","last_clean":"2024-04-26T21:59:51.149237+0000","last_became_active":"2024-04-26T21:59:48.148920+0000","last_became_peered":"2024-04-26T21:59:48.148920+0000","last_unstale":"2024-04-26T21:59:51.149237+0000","last_undegraded":"2024-04-26T21:59:51.149237+0000","last_fullsized":"2024-04-26T21:59:51.149237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:46.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120451+0000","last_change":"2024-04-26T21:59:43.040576+0000","last_active":"2024-04-26T21:59:49.120451+0000","last_peered":"2024-04-26T21:59:49.120451+0000","last_clean":"2024-04-26T21:59:49.120451+0000","last_became_active":"2024-04-26T21:59:43.040217+0000","last_became_peered":"2024-04-26T21:59:43.040217+0000","last_unstale":"2024-04-26T21:59:49.120451+0000","last_undegraded":"2024-04-26T21:59:49.120451+0000","last_fullsized":"2024-04-26T21:59:49.120451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:58.860607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661528+0000","last_change":"2024-04-26T21:59:44.067746+0000","last_active":"2024-04-26T21:59:50.661528+0000","last_peered":"2024-04-26T21:59:50.661528+0000","last_clean":"2024-04-26T21:59:50.661528+0000","last_became_active":"2024-04-26T21:59:44.067434+0000","last_became_peered":"2024-04-26T21:59:44.067434+0000","last_unstale":"2024-04-26T21:59:50.661528+0000","last_undegraded":"2024-04-26T21:59:50.661528+0000","last_fullsized":"2024-04-26T21:59:50.661528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:45.304899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459553+0000","last_change":"2024-04-26T21:59:48.157380+0000","last_active":"2024-04-26T21:59:49.459553+0000","last_peered":"2024-04-26T21:59:49.459553+0000","last_clean":"2024-04-26T21:59:49.459553+0000","last_became_active":"2024-04-26T21:59:48.155887+0000","last_became_peered":"2024-04-26T21:59:48.155887+0000","last_unstale":"2024-04-26T21:59:49.459553+0000","last_undegraded":"2024-04-26T21:59:49.459553+0000","last_fullsized":"2024-04-26T21:59:49.459553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.339191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.117947+0000","last_change":"2024-04-26T21:59:43.036773+0000","last_active":"2024-04-26T21:59:49.117947+0000","last_peered":"2024-04-26T21:59:49.117947+0000","last_clean":"2024-04-26T21:59:49.117947+0000","last_became_active":"2024-04-26T21:59:43.036384+0000","last_became_peered":"2024-04-26T21:59:43.036384+0000","last_unstale":"2024-04-26T21:59:49.117947+0000","last_undegraded":"2024-04-26T21:59:49.117947+0000","last_fullsized":"2024-04-26T21:59:49.117947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:33.841309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661381+0000","last_change":"2024-04-26T21:59:44.043546+0000","last_active":"2024-04-26T21:59:50.661381+0000","last_peered":"2024-04-26T21:59:50.661381+0000","last_clean":"2024-04-26T21:59:50.661381+0000","last_became_active":"2024-04-26T21:59:44.043221+0000","last_became_peered":"2024-04-26T21:59:44.043221+0000","last_unstale":"2024-04-26T21:59:50.661381+0000","last_undegraded":"2024-04-26T21:59:50.661381+0000","last_fullsized":"2024-04-26T21:59:50.661381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:47.436179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723618+0000","last_change":"2024-04-26T21:59:48.156690+0000","last_active":"2024-04-26T21:59:50.723618+0000","last_peered":"2024-04-26T21:59:50.723618+0000","last_clean":"2024-04-26T21:59:50.723618+0000","last_became_active":"2024-04-26T21:59:48.155067+0000","last_became_peered":"2024-04-26T21:59:48.155067+0000","last_unstale":"2024-04-26T21:59:50.723618+0000","last_undegraded":"2024-04-26T21:59:50.723618+0000","last_fullsized":"2024-04-26T21:59:50.723618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:39.366540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719457+0000","last_change":"2024-04-26T21:59:43.050964+0000","last_active":"2024-04-26T21:59:50.719457+0000","last_peered":"2024-04-26T21:59:50.719457+0000","last_clean":"2024-04-26T21:59:50.719457+0000","last_became_active":"2024-04-26T21:59:43.050622+0000","last_became_peered":"2024-04-26T21:59:43.050622+0000","last_unstale":"2024-04-26T21:59:50.719457+0000","last_undegraded":"2024-04-26T21:59:50.719457+0000","last_fullsized":"2024-04-26T21:59:50.719457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:18.246708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120090+0000","last_change":"2024-04-26T21:59:44.044126+0000","last_active":"2024-04-26T21:59:49.120090+0000","last_peered":"2024-04-26T21:59:49.120090+0000","last_clean":"2024-04-26T21:59:49.120090+0000","last_became_active":"2024-04-26T21:59:44.043800+0000","last_became_peered":"2024-04-26T21:59:44.043800+0000","last_unstale":"2024-04-26T21:59:49.120090+0000","last_undegraded":"2024-04-26T21:59:49.120090+0000","last_fullsized":"2024-04-26T21:59:49.120090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:46.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120529+0000","last_change":"2024-04-26T21:59:48.134868+0000","last_active":"2024-04-26T21:59:49.120529+0000","last_peered":"2024-04-26T21:59:49.120529+0000","last_clean":"2024-04-26T21:59:49.120529+0000","last_became_active":"2024-04-26T21:59:48.134545+0000","last_became_peered":"2024-04-26T21:59:48.134545+0000","last_unstale":"2024-04-26T21:59:49.120529+0000","last_undegraded":"2024-04-26T21:59:49.120529+0000","last_fullsized":"2024-04-26T21:59:49.120529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:59.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720077+0000","last_change":"2024-04-26T21:59:43.038912+0000","last_active":"2024-04-26T21:59:50.720077+0000","last_peered":"2024-04-26T21:59:50.720077+0000","last_clean":"2024-04-26T21:59:50.720077+0000","last_became_active":"2024-04-26T21:59:43.038597+0000","last_became_peered":"2024-04-26T21:59:43.038597+0000","last_unstale":"2024-04-26T21:59:50.720077+0000","last_undegraded":"2024-04-26T21:59:50.720077+0000","last_fullsized":"2024-04-26T21:59:50.720077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:37.087595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149414+0000","last_change":"2024-04-26T21:59:44.043297+0000","last_active":"2024-04-26T21:59:51.149414+0000","last_peered":"2024-04-26T21:59:51.149414+0000","last_clean":"2024-04-26T21:59:51.149414+0000","last_became_active":"2024-04-26T21:59:44.042952+0000","last_became_peered":"2024-04-26T21:59:44.042952+0000","last_unstale":"2024-04-26T21:59:51.149414+0000","last_undegraded":"2024-04-26T21:59:51.149414+0000","last_fullsized":"2024-04-26T21:59:51.149414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:18.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719851+0000","last_change":"2024-04-26T21:59:48.155119+0000","last_active":"2024-04-26T21:59:50.719851+0000","last_peered":"2024-04-26T21:59:50.719851+0000","last_clean":"2024-04-26T21:59:50.719851+0000","last_became_active":"2024-04-26T21:59:48.154593+0000","last_became_peered":"2024-04-26T21:59:48.154593+0000","last_unstale":"2024-04-26T21:59:50.719851+0000","last_undegraded":"2024-04-26T21:59:50.719851+0000","last_fullsized":"2024-04-26T21:59:50.719851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:01.896610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.459017+0000","last_change":"2024-04-26T21:59:43.067817+0000","last_active":"2024-04-26T21:59:49.459017+0000","last_peered":"2024-04-26T21:59:49.459017+0000","last_clean":"2024-04-26T21:59:49.459017+0000","last_became_active":"2024-04-26T21:59:43.067450+0000","last_became_peered":"2024-04-26T21:59:43.067450+0000","last_unstale":"2024-04-26T21:59:49.459017+0000","last_undegraded":"2024-04-26T21:59:49.459017+0000","last_fullsized":"2024-04-26T21:59:49.459017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:29.425903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.389031+0000","last_change":"2024-04-26T21:59:44.055085+0000","last_active":"2024-04-26T21:59:50.389031+0000","last_peered":"2024-04-26T21:59:50.389031+0000","last_clean":"2024-04-26T21:59:50.389031+0000","last_became_active":"2024-04-26T21:59:44.054627+0000","last_became_peered":"2024-04-26T21:59:44.054627+0000","last_unstale":"2024-04-26T21:59:50.389031+0000","last_undegraded":"2024-04-26T21:59:50.389031+0000","last_fullsized":"2024-04-26T21:59:50.389031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.234287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722992+0000","last_change":"2024-04-26T21:59:48.147333+0000","last_active":"2024-04-26T21:59:50.722992+0000","last_peered":"2024-04-26T21:59:50.722992+0000","last_clean":"2024-04-26T21:59:50.722992+0000","last_became_active":"2024-04-26T21:59:48.146926+0000","last_became_peered":"2024-04-26T21:59:48.146926+0000","last_unstale":"2024-04-26T21:59:50.722992+0000","last_undegraded":"2024-04-26T21:59:50.722992+0000","last_fullsized":"2024-04-26T21:59:50.722992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661244+0000","last_change":"2024-04-26T21:59:43.050434+0000","last_active":"2024-04-26T21:59:50.661244+0000","last_peered":"2024-04-26T21:59:50.661244+0000","last_clean":"2024-04-26T21:59:50.661244+0000","last_became_active":"2024-04-26T21:59:43.050096+0000","last_became_peered":"2024-04-26T21:59:43.050096+0000","last_unstale":"2024-04-26T21:59:50.661244+0000","last_undegraded":"2024-04-26T21:59:50.661244+0000","last_fullsized":"2024-04-26T21:59:50.661244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:22.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.388887+0000","last_change":"2024-04-26T21:59:44.048302+0000","last_active":"2024-04-26T21:59:50.388887+0000","last_peered":"2024-04-26T21:59:50.388887+0000","last_clean":"2024-04-26T21:59:50.388887+0000","last_became_active":"2024-04-26T21:59:44.047898+0000","last_became_peered":"2024-04-26T21:59:44.047898+0000","last_unstale":"2024-04-26T21:59:50.388887+0000","last_undegraded":"2024-04-26T21:59:50.388887+0000","last_fullsized":"2024-04-26T21:59:50.388887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:55.718558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723553+0000","last_change":"2024-04-26T21:59:48.148025+0000","last_active":"2024-04-26T21:59:50.723553+0000","last_peered":"2024-04-26T21:59:50.723553+0000","last_clean":"2024-04-26T21:59:50.723553+0000","last_became_active":"2024-04-26T21:59:48.147065+0000","last_became_peered":"2024-04-26T21:59:48.147065+0000","last_unstale":"2024-04-26T21:59:50.723553+0000","last_undegraded":"2024-04-26T21:59:50.723553+0000","last_fullsized":"2024-04-26T21:59:50.723553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:23.932501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120192+0000","last_change":"2024-04-26T21:59:44.045186+0000","last_active":"2024-04-26T21:59:49.120192+0000","last_peered":"2024-04-26T21:59:49.120192+0000","last_clean":"2024-04-26T21:59:49.120192+0000","last_became_active":"2024-04-26T21:59:44.044761+0000","last_became_peered":"2024-04-26T21:59:44.044761+0000","last_unstale":"2024-04-26T21:59:49.120192+0000","last_undegraded":"2024-04-26T21:59:49.120192+0000","last_fullsized":"2024-04-26T21:59:49.120192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:32.744412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.722062+0000","last_change":"2024-04-26T21:59:43.068695+0000","last_active":"2024-04-26T21:59:50.722062+0000","last_peered":"2024-04-26T21:59:50.722062+0000","last_clean":"2024-04-26T21:59:50.722062+0000","last_became_active":"2024-04-26T21:59:43.068210+0000","last_became_peered":"2024-04-26T21:59:43.068210+0000","last_unstale":"2024-04-26T21:59:50.722062+0000","last_undegraded":"2024-04-26T21:59:50.722062+0000","last_fullsized":"2024-04-26T21:59:50.722062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:19.914872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661647+0000","last_change":"2024-04-26T21:59:48.154253+0000","last_active":"2024-04-26T21:59:50.661647+0000","last_peered":"2024-04-26T21:59:50.661647+0000","last_clean":"2024-04-26T21:59:50.661647+0000","last_became_active":"2024-04-26T21:59:48.153877+0000","last_became_peered":"2024-04-26T21:59:48.153877+0000","last_unstale":"2024-04-26T21:59:50.661647+0000","last_undegraded":"2024-04-26T21:59:50.661647+0000","last_fullsized":"2024-04-26T21:59:50.661647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:28.163306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120994+0000","last_change":"2024-04-26T21:59:44.052732+0000","last_active":"2024-04-26T21:59:49.120994+0000","last_peered":"2024-04-26T21:59:49.120994+0000","last_clean":"2024-04-26T21:59:49.120994+0000","last_became_active":"2024-04-26T21:59:44.051202+0000","last_became_peered":"2024-04-26T21:59:44.051202+0000","last_unstale":"2024-04-26T21:59:49.120994+0000","last_undegraded":"2024-04-26T21:59:49.120994+0000","last_fullsized":"2024-04-26T21:59:49.120994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:20.887239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118164+0000","last_change":"2024-04-26T21:59:43.054776+0000","last_active":"2024-04-26T21:59:49.118164+0000","last_peered":"2024-04-26T21:59:49.118164+0000","last_clean":"2024-04-26T21:59:49.118164+0000","last_became_active":"2024-04-26T21:59:43.054410+0000","last_became_peered":"2024-04-26T21:59:43.054410+0000","last_unstale":"2024-04-26T21:59:49.118164+0000","last_undegraded":"2024-04-26T21:59:49.118164+0000","last_fullsized":"2024-04-26T21:59:49.118164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:11.675134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661021+0000","last_change":"2024-04-26T21:58:22.546842+0000","last_active":"2024-04-26T21:59:50.661021+0000","last_peered":"2024-04-26T21:59:50.661021+0000","last_clean":"2024-04-26T21:59:50.661021+0000","last_became_active":"2024-04-26T21:58:22.546448+0000","last_became_peered":"2024-04-26T21:58:22.546448+0000","last_unstale":"2024-04-26T21:59:50.661021+0000","last_undegraded":"2024-04-26T21:59:50.661021+0000","last_fullsized":"2024-04-26T21:59:50.661021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_clean_scrub_stamp":"2024-04-26T21:54:29.502062+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:31.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120544+0000","last_change":"2024-04-26T21:59:48.152986+0000","last_active":"2024-04-26T21:59:49.120544+0000","last_peered":"2024-04-26T21:59:49.120544+0000","last_clean":"2024-04-26T21:59:49.120544+0000","last_became_active":"2024-04-26T21:59:48.152606+0000","last_became_peered":"2024-04-26T21:59:48.152606+0000","last_unstale":"2024-04-26T21:59:49.120544+0000","last_undegraded":"2024-04-26T21:59:49.120544+0000","last_fullsized":"2024-04-26T21:59:49.120544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.721974+0000","last_change":"2024-04-26T21:59:44.060086+0000","last_active":"2024-04-26T21:59:50.721974+0000","last_peered":"2024-04-26T21:59:50.721974+0000","last_clean":"2024-04-26T21:59:50.721974+0000","last_became_active":"2024-04-26T21:59:44.059742+0000","last_became_peered":"2024-04-26T21:59:44.059742+0000","last_unstale":"2024-04-26T21:59:50.721974+0000","last_undegraded":"2024-04-26T21:59:50.721974+0000","last_fullsized":"2024-04-26T21:59:50.721974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.678769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661351+0000","last_change":"2024-04-26T21:59:43.034726+0000","last_active":"2024-04-26T21:59:50.661351+0000","last_peered":"2024-04-26T21:59:50.661351+0000","last_clean":"2024-04-26T21:59:50.661351+0000","last_became_active":"2024-04-26T21:59:43.034407+0000","last_became_peered":"2024-04-26T21:59:43.034407+0000","last_unstale":"2024-04-26T21:59:50.661351+0000","last_undegraded":"2024-04-26T21:59:50.661351+0000","last_fullsized":"2024-04-26T21:59:50.661351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:37.722860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149321+0000","last_change":"2024-04-26T21:59:48.136044+0000","last_active":"2024-04-26T21:59:51.149321+0000","last_peered":"2024-04-26T21:59:51.149321+0000","last_clean":"2024-04-26T21:59:51.149321+0000","last_became_active":"2024-04-26T21:59:48.135619+0000","last_became_peered":"2024-04-26T21:59:48.135619+0000","last_unstale":"2024-04-26T21:59:51.149321+0000","last_undegraded":"2024-04-26T21:59:51.149321+0000","last_fullsized":"2024-04-26T21:59:51.149321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:24.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719923+0000","last_change":"2024-04-26T21:59:43.059149+0000","last_active":"2024-04-26T21:59:50.719923+0000","last_peered":"2024-04-26T21:59:50.719923+0000","last_clean":"2024-04-26T21:59:50.719923+0000","last_became_active":"2024-04-26T21:59:43.058816+0000","last_became_peered":"2024-04-26T21:59:43.058816+0000","last_unstale":"2024-04-26T21:59:50.719923+0000","last_undegraded":"2024-04-26T21:59:50.719923+0000","last_fullsized":"2024-04-26T21:59:50.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:04.441843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723197+0000","last_change":"2024-04-26T21:59:44.063544+0000","last_active":"2024-04-26T21:59:50.723197+0000","last_peered":"2024-04-26T21:59:50.723197+0000","last_clean":"2024-04-26T21:59:50.723197+0000","last_became_active":"2024-04-26T21:59:44.063179+0000","last_became_peered":"2024-04-26T21:59:44.063179+0000","last_unstale":"2024-04-26T21:59:50.723197+0000","last_undegraded":"2024-04-26T21:59:50.723197+0000","last_fullsized":"2024-04-26T21:59:50.723197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:44.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120312+0000","last_change":"2024-04-26T21:59:48.131695+0000","last_active":"2024-04-26T21:59:49.120312+0000","last_peered":"2024-04-26T21:59:49.120312+0000","last_clean":"2024-04-26T21:59:49.120312+0000","last_became_active":"2024-04-26T21:59:48.131246+0000","last_became_peered":"2024-04-26T21:59:48.131246+0000","last_unstale":"2024-04-26T21:59:49.120312+0000","last_undegraded":"2024-04-26T21:59:49.120312+0000","last_fullsized":"2024-04-26T21:59:49.120312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:53.128553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T22:00:11.079444+0000","last_change":"2024-04-26T21:59:43.067463+0000","last_active":"2024-04-26T22:00:11.079444+0000","last_peered":"2024-04-26T22:00:11.079444+0000","last_clean":"2024-04-26T22:00:11.079444+0000","last_became_active":"2024-04-26T21:59:43.067009+0000","last_became_peered":"2024-04-26T21:59:43.067009+0000","last_unstale":"2024-04-26T22:00:11.079444+0000","last_undegraded":"2024-04-26T22:00:11.079444+0000","last_fullsized":"2024-04-26T22:00:11.079444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:35.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122236+0000","last_change":"2024-04-26T21:59:44.054781+0000","last_active":"2024-04-26T21:59:49.122236+0000","last_peered":"2024-04-26T21:59:49.122236+0000","last_clean":"2024-04-26T21:59:49.122236+0000","last_became_active":"2024-04-26T21:59:44.049439+0000","last_became_peered":"2024-04-26T21:59:44.049439+0000","last_unstale":"2024-04-26T21:59:49.122236+0000","last_undegraded":"2024-04-26T21:59:49.122236+0000","last_fullsized":"2024-04-26T21:59:49.122236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:50.523835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313357+0000","last_change":"2024-04-26T21:59:48.127776+0000","last_active":"2024-04-26T21:59:50.313357+0000","last_peered":"2024-04-26T21:59:50.313357+0000","last_clean":"2024-04-26T21:59:50.313357+0000","last_became_active":"2024-04-26T21:59:48.127207+0000","last_became_peered":"2024-04-26T21:59:48.127207+0000","last_unstale":"2024-04-26T21:59:50.313357+0000","last_undegraded":"2024-04-26T21:59:50.313357+0000","last_fullsized":"2024-04-26T21:59:50.313357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:23.794371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118341+0000","last_change":"2024-04-26T21:59:43.050320+0000","last_active":"2024-04-26T21:59:49.118341+0000","last_peered":"2024-04-26T21:59:49.118341+0000","last_clean":"2024-04-26T21:59:49.118341+0000","last_became_active":"2024-04-26T21:59:43.049890+0000","last_became_peered":"2024-04-26T21:59:43.049890+0000","last_unstale":"2024-04-26T21:59:49.118341+0000","last_undegraded":"2024-04-26T21:59:49.118341+0000","last_fullsized":"2024-04-26T21:59:49.118341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:53.589776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324595+0000","last_change":"2024-04-26T21:59:44.057851+0000","last_active":"2024-04-26T21:59:50.324595+0000","last_peered":"2024-04-26T21:59:50.324595+0000","last_clean":"2024-04-26T21:59:50.324595+0000","last_became_active":"2024-04-26T21:59:44.057525+0000","last_became_peered":"2024-04-26T21:59:44.057525+0000","last_unstale":"2024-04-26T21:59:50.324595+0000","last_undegraded":"2024-04-26T21:59:50.324595+0000","last_fullsized":"2024-04-26T21:59:50.324595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:30.986251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118072+0000","last_change":"2024-04-26T21:59:48.154962+0000","last_active":"2024-04-26T21:59:49.118072+0000","last_peered":"2024-04-26T21:59:49.118072+0000","last_clean":"2024-04-26T21:59:49.118072+0000","last_became_active":"2024-04-26T21:59:48.154479+0000","last_became_peered":"2024-04-26T21:59:48.154479+0000","last_unstale":"2024-04-26T21:59:49.118072+0000","last_undegraded":"2024-04-26T21:59:49.118072+0000","last_fullsized":"2024-04-26T21:59:49.118072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:30.415432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719716+0000","last_change":"2024-04-26T21:59:43.051186+0000","last_active":"2024-04-26T21:59:50.719716+0000","last_peered":"2024-04-26T21:59:50.719716+0000","last_clean":"2024-04-26T21:59:50.719716+0000","last_became_active":"2024-04-26T21:59:43.050768+0000","last_became_peered":"2024-04-26T21:59:43.050768+0000","last_unstale":"2024-04-26T21:59:50.719716+0000","last_undegraded":"2024-04-26T21:59:50.719716+0000","last_fullsized":"2024-04-26T21:59:50.719716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.237229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719465+0000","last_change":"2024-04-26T21:59:44.039113+0000","last_active":"2024-04-26T21:59:50.719465+0000","last_peered":"2024-04-26T21:59:50.719465+0000","last_clean":"2024-04-26T21:59:50.719465+0000","last_became_active":"2024-04-26T21:59:44.038610+0000","last_became_peered":"2024-04-26T21:59:44.038610+0000","last_unstale":"2024-04-26T21:59:50.719465+0000","last_undegraded":"2024-04-26T21:59:50.719465+0000","last_fullsized":"2024-04-26T21:59:50.719465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:39.450603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.460041+0000","last_change":"2024-04-26T21:59:48.157058+0000","last_active":"2024-04-26T21:59:49.460041+0000","last_peered":"2024-04-26T21:59:49.460041+0000","last_clean":"2024-04-26T21:59:49.460041+0000","last_became_active":"2024-04-26T21:59:48.153897+0000","last_became_peered":"2024-04-26T21:59:48.153897+0000","last_unstale":"2024-04-26T21:59:49.460041+0000","last_undegraded":"2024-04-26T21:59:49.460041+0000","last_fullsized":"2024-04-26T21:59:49.460041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.044493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120593+0000","last_change":"2024-04-26T21:59:43.061101+0000","last_active":"2024-04-26T21:59:49.120593+0000","last_peered":"2024-04-26T21:59:49.120593+0000","last_clean":"2024-04-26T21:59:49.120593+0000","last_became_active":"2024-04-26T21:59:43.060762+0000","last_became_peered":"2024-04-26T21:59:43.060762+0000","last_unstale":"2024-04-26T21:59:49.120593+0000","last_undegraded":"2024-04-26T21:59:49.120593+0000","last_fullsized":"2024-04-26T21:59:49.120593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:07.961535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149273+0000","last_change":"2024-04-26T21:59:44.047437+0000","last_active":"2024-04-26T21:59:51.149273+0000","last_peered":"2024-04-26T21:59:51.149273+0000","last_clean":"2024-04-26T21:59:51.149273+0000","last_became_active":"2024-04-26T21:59:44.047127+0000","last_became_peered":"2024-04-26T21:59:44.047127+0000","last_unstale":"2024-04-26T21:59:51.149273+0000","last_undegraded":"2024-04-26T21:59:51.149273+0000","last_fullsized":"2024-04-26T21:59:51.149273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:12.259965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118210+0000","last_change":"2024-04-26T21:59:48.149822+0000","last_active":"2024-04-26T21:59:49.118210+0000","last_peered":"2024-04-26T21:59:49.118210+0000","last_clean":"2024-04-26T21:59:49.118210+0000","last_became_active":"2024-04-26T21:59:48.149409+0000","last_became_peered":"2024-04-26T21:59:48.149409+0000","last_unstale":"2024-04-26T21:59:49.118210+0000","last_undegraded":"2024-04-26T21:59:49.118210+0000","last_fullsized":"2024-04-26T21:59:49.118210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:12:14.161867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313388+0000","last_change":"2024-04-26T21:59:43.043901+0000","last_active":"2024-04-26T21:59:50.313388+0000","last_peered":"2024-04-26T21:59:50.313388+0000","last_clean":"2024-04-26T21:59:50.313388+0000","last_became_active":"2024-04-26T21:59:43.043572+0000","last_became_peered":"2024-04-26T21:59:43.043572+0000","last_unstale":"2024-04-26T21:59:50.313388+0000","last_undegraded":"2024-04-26T21:59:50.313388+0000","last_fullsized":"2024-04-26T21:59:50.313388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.107295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120801+0000","last_change":"2024-04-26T21:59:44.059451+0000","last_active":"2024-04-26T21:59:49.120801+0000","last_peered":"2024-04-26T21:59:49.120801+0000","last_clean":"2024-04-26T21:59:49.120801+0000","last_became_active":"2024-04-26T21:59:44.056771+0000","last_became_peered":"2024-04-26T21:59:44.056771+0000","last_unstale":"2024-04-26T21:59:49.120801+0000","last_undegraded":"2024-04-26T21:59:49.120801+0000","last_fullsized":"2024-04-26T21:59:49.120801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:53.641182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.118177+0000","last_change":"2024-04-26T21:59:48.130222+0000","last_active":"2024-04-26T21:59:49.118177+0000","last_peered":"2024-04-26T21:59:49.118177+0000","last_clean":"2024-04-26T21:59:49.118177+0000","last_became_active":"2024-04-26T21:59:48.129379+0000","last_became_peered":"2024-04-26T21:59:48.129379+0000","last_unstale":"2024-04-26T21:59:49.118177+0000","last_undegraded":"2024-04-26T21:59:49.118177+0000","last_fullsized":"2024-04-26T21:59:49.118177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:26.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719646+0000","last_change":"2024-04-26T21:59:43.035503+0000","last_active":"2024-04-26T21:59:50.719646+0000","last_peered":"2024-04-26T21:59:50.719646+0000","last_clean":"2024-04-26T21:59:50.719646+0000","last_became_active":"2024-04-26T21:59:43.035180+0000","last_became_peered":"2024-04-26T21:59:43.035180+0000","last_unstale":"2024-04-26T21:59:50.719646+0000","last_undegraded":"2024-04-26T21:59:50.719646+0000","last_fullsized":"2024-04-26T21:59:50.719646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:45.696707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661312+0000","last_change":"2024-04-26T21:59:44.054444+0000","last_active":"2024-04-26T21:59:50.661312+0000","last_peered":"2024-04-26T21:59:50.661312+0000","last_clean":"2024-04-26T21:59:50.661312+0000","last_became_active":"2024-04-26T21:59:44.054035+0000","last_became_peered":"2024-04-26T21:59:44.054035+0000","last_unstale":"2024-04-26T21:59:50.661312+0000","last_undegraded":"2024-04-26T21:59:50.661312+0000","last_fullsized":"2024-04-26T21:59:50.661312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:04.905286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.719879+0000","last_change":"2024-04-26T21:59:48.141551+0000","last_active":"2024-04-26T21:59:50.719879+0000","last_peered":"2024-04-26T21:59:50.719879+0000","last_clean":"2024-04-26T21:59:50.719879+0000","last_became_active":"2024-04-26T21:59:48.141166+0000","last_became_peered":"2024-04-26T21:59:48.141166+0000","last_unstale":"2024-04-26T21:59:50.719879+0000","last_undegraded":"2024-04-26T21:59:50.719879+0000","last_fullsized":"2024-04-26T21:59:50.719879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:38.810387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313994+0000","last_change":"2024-04-26T21:59:43.059420+0000","last_active":"2024-04-26T21:59:50.313994+0000","last_peered":"2024-04-26T21:59:50.313994+0000","last_clean":"2024-04-26T21:59:50.313994+0000","last_became_active":"2024-04-26T21:59:43.059033+0000","last_became_peered":"2024-04-26T21:59:43.059033+0000","last_unstale":"2024-04-26T21:59:50.313994+0000","last_undegraded":"2024-04-26T21:59:50.313994+0000","last_fullsized":"2024-04-26T21:59:50.313994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:57.633543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661718+0000","last_change":"2024-04-26T21:59:44.045932+0000","last_active":"2024-04-26T21:59:50.661718+0000","last_peered":"2024-04-26T21:59:50.661718+0000","last_clean":"2024-04-26T21:59:50.661718+0000","last_became_active":"2024-04-26T21:59:44.044775+0000","last_became_peered":"2024-04-26T21:59:44.044775+0000","last_unstale":"2024-04-26T21:59:50.661718+0000","last_undegraded":"2024-04-26T21:59:50.661718+0000","last_fullsized":"2024-04-26T21:59:50.661718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:05.719002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.323950+0000","last_change":"2024-04-26T21:59:48.151182+0000","last_active":"2024-04-26T21:59:50.323950+0000","last_peered":"2024-04-26T21:59:50.323950+0000","last_clean":"2024-04-26T21:59:50.323950+0000","last_became_active":"2024-04-26T21:59:48.150815+0000","last_became_peered":"2024-04-26T21:59:48.150815+0000","last_unstale":"2024-04-26T21:59:50.323950+0000","last_undegraded":"2024-04-26T21:59:50.323950+0000","last_fullsized":"2024-04-26T21:59:50.323950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.630163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120863+0000","last_change":"2024-04-26T21:59:43.039852+0000","last_active":"2024-04-26T21:59:49.120863+0000","last_peered":"2024-04-26T21:59:49.120863+0000","last_clean":"2024-04-26T21:59:49.120863+0000","last_became_active":"2024-04-26T21:59:43.039499+0000","last_became_peered":"2024-04-26T21:59:43.039499+0000","last_unstale":"2024-04-26T21:59:49.120863+0000","last_undegraded":"2024-04-26T21:59:49.120863+0000","last_fullsized":"2024-04-26T21:59:49.120863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:14.599270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661485+0000","last_change":"2024-04-26T21:59:44.057285+0000","last_active":"2024-04-26T21:59:50.661485+0000","last_peered":"2024-04-26T21:59:50.661485+0000","last_clean":"2024-04-26T21:59:50.661485+0000","last_became_active":"2024-04-26T21:59:44.056947+0000","last_became_peered":"2024-04-26T21:59:44.056947+0000","last_unstale":"2024-04-26T21:59:50.661485+0000","last_undegraded":"2024-04-26T21:59:50.661485+0000","last_fullsized":"2024-04-26T21:59:50.661485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:58.242075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.119912+0000","last_change":"2024-04-26T21:59:48.131624+0000","last_active":"2024-04-26T21:59:49.119912+0000","last_peered":"2024-04-26T21:59:49.119912+0000","last_clean":"2024-04-26T21:59:49.119912+0000","last_became_active":"2024-04-26T21:59:48.131193+0000","last_became_peered":"2024-04-26T21:59:48.131193+0000","last_unstale":"2024-04-26T21:59:49.119912+0000","last_undegraded":"2024-04-26T21:59:49.119912+0000","last_fullsized":"2024-04-26T21:59:49.119912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:52.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723555+0000","last_change":"2024-04-26T21:59:43.047723+0000","last_active":"2024-04-26T21:59:50.723555+0000","last_peered":"2024-04-26T21:59:50.723555+0000","last_clean":"2024-04-26T21:59:50.723555+0000","last_became_active":"2024-04-26T21:59:43.045876+0000","last_became_peered":"2024-04-26T21:59:43.045876+0000","last_unstale":"2024-04-26T21:59:50.723555+0000","last_undegraded":"2024-04-26T21:59:50.723555+0000","last_fullsized":"2024-04-26T21:59:50.723555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:37.705490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.661733+0000","last_change":"2024-04-26T21:59:44.040723+0000","last_active":"2024-04-26T21:59:50.661733+0000","last_peered":"2024-04-26T21:59:50.661733+0000","last_clean":"2024-04-26T21:59:50.661733+0000","last_became_active":"2024-04-26T21:59:44.040389+0000","last_became_peered":"2024-04-26T21:59:44.040389+0000","last_unstale":"2024-04-26T21:59:50.661733+0000","last_undegraded":"2024-04-26T21:59:50.661733+0000","last_fullsized":"2024-04-26T21:59:50.661733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:05.120695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149410+0000","last_change":"2024-04-26T21:59:44.041027+0000","last_active":"2024-04-26T21:59:51.149410+0000","last_peered":"2024-04-26T21:59:51.149410+0000","last_clean":"2024-04-26T21:59:51.149410+0000","last_became_active":"2024-04-26T21:59:44.040705+0000","last_became_peered":"2024-04-26T21:59:44.040705+0000","last_unstale":"2024-04-26T21:59:51.149410+0000","last_undegraded":"2024-04-26T21:59:51.149410+0000","last_fullsized":"2024-04-26T21:59:51.149410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:11.114850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313739+0000","last_change":"2024-04-26T21:59:44.040916+0000","last_active":"2024-04-26T21:59:50.313739+0000","last_peered":"2024-04-26T21:59:50.313739+0000","last_clean":"2024-04-26T21:59:50.313739+0000","last_became_active":"2024-04-26T21:59:44.040568+0000","last_became_peered":"2024-04-26T21:59:44.040568+0000","last_unstale":"2024-04-26T21:59:50.313739+0000","last_undegraded":"2024-04-26T21:59:50.313739+0000","last_fullsized":"2024-04-26T21:59:50.313739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:04.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313347+0000","last_change":"2024-04-26T21:59:44.046676+0000","last_active":"2024-04-26T21:59:50.313347+0000","last_peered":"2024-04-26T21:59:50.313347+0000","last_clean":"2024-04-26T21:59:50.313347+0000","last_became_active":"2024-04-26T21:59:44.046339+0000","last_became_peered":"2024-04-26T21:59:44.046339+0000","last_unstale":"2024-04-26T21:59:50.313347+0000","last_undegraded":"2024-04-26T21:59:50.313347+0000","last_fullsized":"2024-04-26T21:59:50.313347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:51.964558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.324145+0000","last_change":"2024-04-26T21:59:44.064635+0000","last_active":"2024-04-26T21:59:50.324145+0000","last_peered":"2024-04-26T21:59:50.324145+0000","last_clean":"2024-04-26T21:59:50.324145+0000","last_became_active":"2024-04-26T21:59:44.064174+0000","last_became_peered":"2024-04-26T21:59:44.064174+0000","last_unstale":"2024-04-26T21:59:50.324145+0000","last_undegraded":"2024-04-26T21:59:50.324145+0000","last_fullsized":"2024-04-26T21:59:50.324145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:59.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723241+0000","last_change":"2024-04-26T21:59:44.051872+0000","last_active":"2024-04-26T21:59:50.723241+0000","last_peered":"2024-04-26T21:59:50.723241+0000","last_clean":"2024-04-26T21:59:50.723241+0000","last_became_active":"2024-04-26T21:59:44.051539+0000","last_became_peered":"2024-04-26T21:59:44.051539+0000","last_unstale":"2024-04-26T21:59:50.723241+0000","last_undegraded":"2024-04-26T21:59:50.723241+0000","last_fullsized":"2024-04-26T21:59:50.723241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:15.622222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149504+0000","last_change":"2024-04-26T21:59:44.044024+0000","last_active":"2024-04-26T21:59:51.149504+0000","last_peered":"2024-04-26T21:59:51.149504+0000","last_clean":"2024-04-26T21:59:51.149504+0000","last_became_active":"2024-04-26T21:59:44.043718+0000","last_became_peered":"2024-04-26T21:59:44.043718+0000","last_unstale":"2024-04-26T21:59:51.149504+0000","last_undegraded":"2024-04-26T21:59:51.149504+0000","last_fullsized":"2024-04-26T21:59:51.149504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:35.028538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.313921+0000","last_change":"2024-04-26T21:59:44.037432+0000","last_active":"2024-04-26T21:59:50.313921+0000","last_peered":"2024-04-26T21:59:50.313921+0000","last_clean":"2024-04-26T21:59:50.313921+0000","last_became_active":"2024-04-26T21:59:44.037086+0000","last_became_peered":"2024-04-26T21:59:44.037086+0000","last_unstale":"2024-04-26T21:59:50.313921+0000","last_undegraded":"2024-04-26T21:59:50.313921+0000","last_fullsized":"2024-04-26T21:59:50.313921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:32.828866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.122540+0000","last_change":"2024-04-26T21:59:44.055253+0000","last_active":"2024-04-26T21:59:49.122540+0000","last_peered":"2024-04-26T21:59:49.122540+0000","last_clean":"2024-04-26T21:59:49.122540+0000","last_became_active":"2024-04-26T21:59:44.054329+0000","last_became_peered":"2024-04-26T21:59:44.054329+0000","last_unstale":"2024-04-26T21:59:49.122540+0000","last_undegraded":"2024-04-26T21:59:49.122540+0000","last_fullsized":"2024-04-26T21:59:49.122540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:02.949272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.120075+0000","last_change":"2024-04-26T21:59:44.060213+0000","last_active":"2024-04-26T21:59:49.120075+0000","last_peered":"2024-04-26T21:59:49.120075+0000","last_clean":"2024-04-26T21:59:49.120075+0000","last_became_active":"2024-04-26T21:59:44.059987+0000","last_became_peered":"2024-04-26T21:59:44.059987+0000","last_unstale":"2024-04-26T21:59:49.120075+0000","last_undegraded":"2024-04-26T21:59:49.120075+0000","last_fullsized":"2024-04-26T21:59:49.120075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:23.184373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:49.121153+0000","last_change":"2024-04-26T21:59:44.046779+0000","last_active":"2024-04-26T21:59:49.121153+0000","last_peered":"2024-04-26T21:59:49.121153+0000","last_clean":"2024-04-26T21:59:49.121153+0000","last_became_active":"2024-04-26T21:59:44.046304+0000","last_became_peered":"2024-04-26T21:59:44.046304+0000","last_unstale":"2024-04-26T21:59:49.121153+0000","last_undegraded":"2024-04-26T21:59:49.121153+0000","last_fullsized":"2024-04-26T21:59:49.121153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:59.738908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.720000+0000","last_change":"2024-04-26T21:59:44.055284+0000","last_active":"2024-04-26T21:59:50.720000+0000","last_peered":"2024-04-26T21:59:50.720000+0000","last_clean":"2024-04-26T21:59:50.720000+0000","last_became_active":"2024-04-26T21:59:44.054897+0000","last_became_peered":"2024-04-26T21:59:44.054897+0000","last_unstale":"2024-04-26T21:59:50.720000+0000","last_undegraded":"2024-04-26T21:59:50.720000+0000","last_fullsized":"2024-04-26T21:59:50.720000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:57.942834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:50.723093+0000","last_change":"2024-04-26T21:59:44.060516+0000","last_active":"2024-04-26T21:59:50.723093+0000","last_peered":"2024-04-26T21:59:50.723093+0000","last_clean":"2024-04-26T21:59:50.723093+0000","last_became_active":"2024-04-26T21:59:44.059564+0000","last_became_peered":"2024-04-26T21:59:44.059564+0000","last_unstale":"2024-04-26T21:59:50.723093+0000","last_undegraded":"2024-04-26T21:59:50.723093+0000","last_fullsized":"2024-04-26T21:59:50.723093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:43.784018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T21:59:51.149377+0000","last_change":"2024-04-26T21:59:44.043564+0000","last_active":"2024-04-26T21:59:51.149377+0000","last_peered":"2024-04-26T21:59:51.149377+0000","last_clean":"2024-04-26T21:59:51.149377+0000","last_became_active":"2024-04-26T21:59:44.043254+0000","last_became_peered":"2024-04-26T21:59:44.043254+0000","last_unstale":"2024-04-26T21:59:51.149377+0000","last_undegraded":"2024-04-26T21:59:51.149377+0000","last_fullsized":"2024-04-26T21:59:51.149377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:12.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":11,"num_write":53,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":13920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1076113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037796,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":486409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1080851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462959,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":485551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":481799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":481813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":481731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1605632,"data_stored":1076355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:14.893 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:00:14.894 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T22:00:14.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T22:00:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:15 smithi012 ceph-mon[28184]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:15 smithi012 ceph-mon[28184]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:15.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:15 smithi154 ceph-mon[32578]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:15.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:15 smithi154 ceph-mon[32578]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:15 smithi077 ceph-mon[32280]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:15 smithi077 ceph-mon[32280]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:00:16.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:16 smithi012 ceph-mon[28184]: pgmap v336: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-04-26T22:00:16.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:16 smithi154 ceph-mon[32578]: pgmap v336: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-04-26T22:00:16.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:16 smithi077 ceph-mon[32280]: pgmap v336: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 -- 172.21.15.12:0/139637792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 msgr2=0x7f4144108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 --2- 172.21.15.12:0/139637792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144108c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4130009990 tx=0x7f4130009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 -- 172.21.15.12:0/139637792 shutdown_connections 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 --2- 172.21.15.12:0/139637792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 0x7f414410b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 --2- 172.21.15.12:0/139637792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 --2- 172.21.15.12:0/139637792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144108c50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 -- 172.21.15.12:0/139637792 >> 172.21.15.12:0/139637792 conn(0x7f41440fd940 msgr2=0x7f41440ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 -- 172.21.15.12:0/139637792 shutdown_connections 2024-04-26T22:00:18.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.066+0000 7f414c600700 1 -- 172.21.15.12:0/139637792 wait complete. 2024-04-26T22:00:18.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.067+0000 7f414c600700 1 Processor -- start 2024-04-26T22:00:18.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 -- start start 2024-04-26T22:00:18.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 0x7f41441a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f414410efb0 con 0x7f4144109190 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f414410ee30 con 0x7f4144106860 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414c600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f414410f130 con 0x7f4144103f30 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f4149c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f4149200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f4149c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43600/0 (socket says 172.21.15.12:43600) 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f4149c00700 1 -- 172.21.15.12:0/2751208093 learned_addr learned my addr 172.21.15.12:0/2751208093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:18.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.068+0000 7f414a600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 0x7f41441a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 -- 172.21.15.12:0/2751208093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 msgr2=0x7f4144199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 -- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 msgr2=0x7f41441a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 0x7f41441a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 -- 172.21.15.12:0/2751208093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441a11b0 con 0x7f4144106860 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149c00700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.069+0000 7f4149200700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144199a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4130000c00 tx=0x7f41300057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.070+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130026070 con 0x7f4144106860 2024-04-26T22:00:18.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.070+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441a1440 con 0x7f4144106860 2024-04-26T22:00:18.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.070+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4130021410 con 0x7f4144106860 2024-04-26T22:00:18.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.070+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130017650 con 0x7f4144106860 2024-04-26T22:00:18.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.070+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a18f0 con 0x7f4144106860 2024-04-26T22:00:18.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.071+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4144051960 con 0x7f4144106860 2024-04-26T22:00:18.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.073+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4130029070 con 0x7f4144106860 2024-04-26T22:00:18.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.074+0000 7f4142000700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 0x7f412c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.075+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f41300d15f0 con 0x7f4144106860 2024-04-26T22:00:18.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.075+0000 7f4149c00700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 0x7f412c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.075+0000 7f4149c00700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 0x7f412c0a6240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4138011590 tx=0x7f4138009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.079+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f413005e060 con 0x7f4144106860 2024-04-26T22:00:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.410+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f414402e490 con 0x7f4144106860 2024-04-26T22:00:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.417+0000 7f4142000700 1 -- 172.21.15.12:0/2751208093 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4130013070 con 0x7f4144106860 2024-04-26T22:00:18.418 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-04-26T22:00:18.418 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDyIyxmUQWpGBAAvrhCLpueOpUFSeEAbRuOEQ== 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 msgr2=0x7f412c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 0x7f412c0a6240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4138011590 tx=0x7f4138009260 comp rx=0 tx=0).stop 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 msgr2=0x7f4144199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144199a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4130000c00 tx=0x7f41300057a0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 shutdown_connections 2024-04-26T22:00:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4144109190 0x7f41441a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f412c0a3d80 0x7f412c0a6240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4144103f30 0x7f4144199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 --2- 172.21.15.12:0/2751208093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4144106860 0x7f4144199a90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 >> 172.21.15.12:0/2751208093 conn(0x7f41440fd940 msgr2=0x7f4144104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 shutdown_connections 2024-04-26T22:00:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:18.420+0000 7f414c600700 1 -- 172.21.15.12:0/2751208093 wait complete. 2024-04-26T22:00:18.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:18 smithi012 ceph-mon[28184]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-04-26T22:00:18.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:18 smithi154 ceph-mon[32578]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-04-26T22:00:18.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:18 smithi077 ceph-mon[32280]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-04-26T22:00:19.369 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:00:19.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T22:00:19.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T22:00:19.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:19 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2751208093' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:19 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:19 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:00:19.496 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T22:00:19.496 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T22:00:19.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T22:00:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:19 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2751208093' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:19 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:19 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:00:19.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:19 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2751208093' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:19 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T22:00:19.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:19 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T22:00:20.009 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:20.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:20 smithi077 ceph-mon[32280]: pgmap v338: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.4 KiB/s wr, 3 op/s 2024-04-26T22:00:20.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:20 smithi012 ceph-mon[28184]: pgmap v338: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.4 KiB/s wr, 3 op/s 2024-04-26T22:00:20.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:20 smithi154 ceph-mon[32578]: pgmap v338: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.4 KiB/s wr, 3 op/s 2024-04-26T22:00:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 -- 172.21.15.12:0/2507096425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 msgr2=0x7fbcd406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/2507096425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd406b590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009990 tx=0x7fbcc0009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 -- 172.21.15.12:0/2507096425 shutdown_connections 2024-04-26T22:00:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/2507096425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/2507096425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 0x7fbcd4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/2507096425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd406b590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 -- 172.21.15.12:0/2507096425 >> 172.21.15.12:0/2507096425 conn(0x7fbcd40792d0 msgr2=0x7fbcd40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 -- 172.21.15.12:0/2507096425 shutdown_connections 2024-04-26T22:00:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.333+0000 7fbcd8a00700 1 -- 172.21.15.12:0/2507096425 wait complete. 2024-04-26T22:00:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.335+0000 7fbcd8a00700 1 Processor -- start 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 -- start start 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 0x7fbcd41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4113620 con 0x7fbcd41094a0 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41134a0 con 0x7fbcd406bad0 2024-04-26T22:00:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.336+0000 7fbcd8a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41137a0 con 0x7fbcd406b130 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd2800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 0x7fbcd41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:36786/0 (socket says 172.21.15.12:36786) 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 -- 172.21.15.12:0/923606383 learned_addr learned my addr 172.21.15.12:0/923606383 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1e00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 -- 172.21.15.12:0/923606383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 msgr2=0x7fbcd419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 -- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 msgr2=0x7fbcd41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 0x7fbcd41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.337+0000 7fbcd1400700 1 -- 172.21.15.12:0/923606383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd41a0d50 con 0x7fbcd406bad0 2024-04-26T22:00:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.338+0000 7fbcd1400700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.338+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0027070 con 0x7fbcd406bad0 2024-04-26T22:00:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.338+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc0022470 con 0x7fbcd406bad0 2024-04-26T22:00:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.338+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd41a0fe0 con 0x7fbcd406bad0 2024-04-26T22:00:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.338+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0017650 con 0x7fbcd406bad0 2024-04-26T22:00:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.339+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41a88c0 con 0x7fbcd406bad0 2024-04-26T22:00:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.340+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd406a110 con 0x7fbcd406bad0 2024-04-26T22:00:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.342+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbcc002a070 con 0x7fbcd406bad0 2024-04-26T22:00:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.343+0000 7fbccaa00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 0x7fbcb80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.343+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbcc00d15d0 con 0x7fbcd406bad0 2024-04-26T22:00:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.343+0000 7fbcd1e00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 0x7fbcb80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.344+0000 7fbcd1e00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 0x7fbcb80a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009bd0 tx=0x7fbcc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.345+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbcc0013070 con 0x7fbcd406bad0 2024-04-26T22:00:22.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.647+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbcd402e490 con 0x7fbcd406bad0 2024-04-26T22:00:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.653+0000 7fbccaa00700 1 -- 172.21.15.12:0/923606383 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264993 (secure 0 0 0) 0x7fbcc009dc40 con 0x7fbcd406bad0 2024-04-26T22:00:22.655 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 msgr2=0x7fbcb80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 0x7fbcb80a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009bd0 tx=0x7fbcc4009260 comp rx=0 tx=0).stop 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 msgr2=0x7fbcd419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc00057c0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 shutdown_connections 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbcd41094a0 0x7fbcd41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbcb80a3cd0 0x7fbcb80a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbcd406b130 0x7fbcd419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 --2- 172.21.15.12:0/923606383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd406bad0 0x7fbcd419c2d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 >> 172.21.15.12:0/923606383 conn(0x7fbcd40792d0 msgr2=0x7fbcd410e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 shutdown_connections 2024-04-26T22:00:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:22.661+0000 7fbcd8a00700 1 -- 172.21.15.12:0/923606383 wait complete. 2024-04-26T22:00:22.680 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:22 smithi012 ceph-mon[28184]: pgmap v339: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-04-26T22:00:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:22 smithi077 ceph-mon[32280]: pgmap v339: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-04-26T22:00:22.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:22 smithi154 ceph-mon[32578]: pgmap v339: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-04-26T22:00:23.693 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/923606383' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:00:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3660687894},{"type":"v1","addr":"172.21.15.12:6801","nonce":3660687894}]},"active_addr":"172.21.15.12:6801/3660687894","active_change":"2024-04-26T21:51:42.978788+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":36130226}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":505816482}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1038496765}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2706265351}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":53237046}]}]} 2024-04-26T22:00:23.702 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T22:00:23.702 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T22:00:23.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T22:00:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/923606383' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:00:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/923606383' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T22:00:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:24 smithi077 ceph-mon[32280]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-04-26T22:00:24.753 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:24 smithi012 ceph-mon[28184]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-04-26T22:00:24.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:24 smithi154 ceph-mon[32578]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 -- 172.21.15.12:0/2823364613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 msgr2=0x7f28701059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 --2- 172.21.15.12:0/2823364613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701059f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f285c0099c0 tx=0x7f285c009cd0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 -- 172.21.15.12:0/2823364613 shutdown_connections 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 --2- 172.21.15.12:0/2823364613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2870105f30 0x7f28701123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 --2- 172.21.15.12:0/2823364613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f287010b620 0x7f287010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 --2- 172.21.15.12:0/2823364613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701059f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.616+0000 7f2876000700 1 -- 172.21.15.12:0/2823364613 >> 172.21.15.12:0/2823364613 conn(0x7f2870101040 msgr2=0x7f2870103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.617+0000 7f2876000700 1 -- 172.21.15.12:0/2823364613 shutdown_connections 2024-04-26T22:00:26.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.617+0000 7f2876000700 1 -- 172.21.15.12:0/2823364613 wait complete. 2024-04-26T22:00:26.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.618+0000 7f2876000700 1 Processor -- start 2024-04-26T22:00:26.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.618+0000 7f2876000700 1 -- start start 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 0x7f28701a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 0x7f287019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2870115730 con 0x7f287010b620 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28701155b0 con 0x7f2870105570 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2876000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28701158b0 con 0x7f2870105f30 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2875600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 0x7f287019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f286fe00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 0x7f28701a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2875600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 0x7f287019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39662/0 (socket says 172.21.15.12:39662) 2024-04-26T22:00:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2874c00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.619+0000 7f2875600700 1 -- 172.21.15.12:0/1746128261 learned_addr learned my addr 172.21.15.12:0/1746128261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 -- 172.21.15.12:0/1746128261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 msgr2=0x7f28701a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 -- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 msgr2=0x7f287019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 0x7f287019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 -- 172.21.15.12:0/1746128261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287019c150 con 0x7f2870105f30 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f2874c00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286fe00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 0x7f28701a1d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b580 tx=0x7f285c0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c026070 con 0x7f2870105f30 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f285c00f460 con 0x7f2870105f30 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.620+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c009e60 con 0x7f2870105f30 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.621+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287019c3e0 con 0x7f2870105f30 2024-04-26T22:00:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.621+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28701a8700 con 0x7f2870105f30 2024-04-26T22:00:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.622+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2870106c90 con 0x7f2870105f30 2024-04-26T22:00:26.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.624+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f285c029070 con 0x7f2870105f30 2024-04-26T22:00:26.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.625+0000 7f286d600700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 0x7f28580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.625+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f285c0d0e50 con 0x7f2870105f30 2024-04-26T22:00:26.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.626+0000 7f2874c00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 0x7f28580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.626+0000 7f2874c00700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 0x7f28580a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f287019cb30 tx=0x7f286400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.628+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f285c013070 con 0x7f2870105f30 2024-04-26T22:00:26.693 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:26 smithi154 ceph-mon[32578]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-26T22:00:26.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:26 smithi077 ceph-mon[32280]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-26T22:00:26.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:26 smithi012 ceph-mon[28184]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-26T22:00:26.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.862+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f287019cf20 con 0x7f2870105f30 2024-04-26T22:00:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.864+0000 7f286d600700 1 -- 172.21.15.12:0/1746128261 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7f285c09d4c0 con 0x7f2870105f30 2024-04-26T22:00:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:26.866 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 msgr2=0x7f28580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 0x7f28580a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f287019cb30 tx=0x7f286400b410 comp rx=0 tx=0).stop 2024-04-26T22:00:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 msgr2=0x7f28701a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 0x7f28701a1d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b580 tx=0x7f285c0055f0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 shutdown_connections 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f287010b620 0x7f287019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2870105570 0x7f28701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f28580a3d50 0x7f28580a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 --2- 172.21.15.12:0/1746128261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2870105f30 0x7f28701a1d70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 >> 172.21.15.12:0/1746128261 conn(0x7f2870101040 msgr2=0x7f2870102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 shutdown_connections 2024-04-26T22:00:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:26.867+0000 7f2876000700 1 -- 172.21.15.12:0/1746128261 wait complete. 2024-04-26T22:00:27.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1746128261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:27.833 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1746128261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:27.835 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T22:00:27.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T22:00:27.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1746128261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:28.292 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:28.673 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:28 smithi012 ceph-mon[28184]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:28.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:28 smithi077 ceph-mon[32280]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:28.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:28 smithi154 ceph-mon[32578]: pgmap v342: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 -- 172.21.15.12:0/220539254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 msgr2=0x7fa07410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 --2- 172.21.15.12:0/220539254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07410f6b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa060009990 tx=0x7fa060009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 -- 172.21.15.12:0/220539254 shutdown_connections 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 --2- 172.21.15.12:0/220539254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa074112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 --2- 172.21.15.12:0/220539254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 0x7fa074075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 --2- 172.21.15.12:0/220539254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07410f6b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 -- 172.21.15.12:0/220539254 >> 172.21.15.12:0/220539254 conn(0x7fa0740feea0 msgr2=0x7fa0741012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:30.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 -- 172.21.15.12:0/220539254 shutdown_connections 2024-04-26T22:00:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.586+0000 7fa073e00700 1 -- 172.21.15.12:0/220539254 wait complete. 2024-04-26T22:00:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.587+0000 7fa073e00700 1 Processor -- start 2024-04-26T22:00:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 -- start start 2024-04-26T22:00:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 0x7fa07419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa0741a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741157a0 con 0x7fa074076110 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa074115620 con 0x7fa07410fbf0 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa073e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa074115920 con 0x7fa0740757f0 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa072000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa072000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49622/0 (socket says 172.21.15.12:49622) 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa072000700 1 -- 172.21.15.12:0/3310754792 learned_addr learned my addr 172.21.15.12:0/3310754792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.588+0000 7fa072a00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 0x7fa07419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:30.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa073400700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa0741a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 -- 172.21.15.12:0/3310754792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 msgr2=0x7fa07419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 0x7fa07419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 -- 172.21.15.12:0/3310754792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 msgr2=0x7fa0741a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa0741a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 -- 172.21.15.12:0/3310754792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0741a0d20 con 0x7fa074076110 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa073400700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa0741a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.589+0000 7fa072000700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa060000c00 tx=0x7fa0600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.590+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060027070 con 0x7fa074076110 2024-04-26T22:00:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.590+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa060022470 con 0x7fa074076110 2024-04-26T22:00:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.590+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741a0fb0 con 0x7fa074076110 2024-04-26T22:00:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.590+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060017650 con 0x7fa074076110 2024-04-26T22:00:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.591+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741a8890 con 0x7fa074076110 2024-04-26T22:00:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.592+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07406a110 con 0x7fa074076110 2024-04-26T22:00:30.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.595+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa06002a070 con 0x7fa074076110 2024-04-26T22:00:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.596+0000 7fa06b400700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 0x7fa0580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:30.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.596+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa0600d14b0 con 0x7fa074076110 2024-04-26T22:00:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.597+0000 7fa072a00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 0x7fa0580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:30.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.597+0000 7fa072a00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 0x7fa0580a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa064009c60 tx=0x7fa064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:30.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.601+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa06005f060 con 0x7fa074076110 2024-04-26T22:00:30.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:30 smithi012 ceph-mon[28184]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:30.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:30 smithi077 ceph-mon[32280]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.839+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa07402e490 con 0x7fa074076110 2024-04-26T22:00:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.842+0000 7fa06b400700 1 -- 172.21.15.12:0/3310754792 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24081 (secure 0 0 0) 0x7fa060013070 con 0x7fa074076110 2024-04-26T22:00:30.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:00:30.844 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T21:59:48.588198+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 msgr2=0x7fa0580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 0x7fa0580a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa064009c60 tx=0x7fa064009260 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 msgr2=0x7fa07419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa060000c00 tx=0x7fa0600057c0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 shutdown_connections 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa07410fbf0 0x7fa0741a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa0580a3cd0 0x7fa0580a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0740757f0 0x7fa07419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 --2- 172.21.15.12:0/3310754792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa074076110 0x7fa07419c250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.846+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 >> 172.21.15.12:0/3310754792 conn(0x7fa0740feea0 msgr2=0x7fa0740ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:30.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.847+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 shutdown_connections 2024-04-26T22:00:30.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:30.847+0000 7fa073e00700 1 -- 172.21.15.12:0/3310754792 wait complete. 2024-04-26T22:00:30.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:30 smithi154 ceph-mon[32578]: pgmap v343: 129 pgs: 129 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T22:00:31.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T22:00:31.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T22:00:31.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T22:00:31.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T22:00:31.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T22:00:31.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T22:00:31.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T22:00:31.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T22:00:31.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T22:00:31.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T22:00:31.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T22:00:31.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T22:00:31.735 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:31 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3310754792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:31.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:31 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3310754792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:31 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3310754792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:00:32.475 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:32 smithi012 ceph-mon[28184]: pgmap v344: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T22:00:32.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:32 smithi077 ceph-mon[32280]: pgmap v344: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T22:00:32.827 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:32.862 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:32.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:32 smithi154 ceph-mon[32578]: pgmap v344: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T22:00:33.530 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:34.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:34 smithi012 ceph-mon[28184]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:34.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:34 smithi077 ceph-mon[32280]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:34.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:34 smithi154 ceph-mon[32578]: pgmap v345: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:36.236 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.237 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.239 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.243 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.244 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.245 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.254 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:00:36.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:36 smithi012 ceph-mon[28184]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:36.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:36 smithi154 ceph-mon[32578]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:36.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:36 smithi077 ceph-mon[32280]: pgmap v346: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:38.752 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:38 smithi012 ceph-mon[28184]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:38.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:38 smithi154 ceph-mon[32578]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:38 smithi077 ceph-mon[32280]: pgmap v347: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:40.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:40 smithi154 ceph-mon[32578]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:40.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:40 smithi012 ceph-mon[28184]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:40.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:40 smithi077 ceph-mon[32280]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:00:42.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:42 smithi012 ceph-mon[28184]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:42.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:42 smithi154 ceph-mon[32578]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:42.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:42 smithi077 ceph-mon[32280]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:00:43.804 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-26T22:00:43.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-26T22:00:43.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:00:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T22:00:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-04-26T22:00:43.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-04-26T22:00:44.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:00:44.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T22:00:44.779 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:00:44.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T22:00:44.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:00:44.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:00:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:45.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:45 smithi154 ceph-mon[32578]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:00:45.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:45 smithi012 ceph-mon[28184]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:00:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:45 smithi077 ceph-mon[32280]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T22:00:46.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:46 smithi012 ceph-mon[28184]: pgmap v353: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:46 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:46 smithi077 ceph-mon[32280]: pgmap v353: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:46 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:47.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:46 smithi154 ceph-mon[32578]: pgmap v353: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:46 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.096+0000 7ff123200700 1 -- 172.21.15.12:0/4067161044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 msgr2=0x7ff124102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.096+0000 7ff123200700 1 --2- 172.21.15.12:0/4067161044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff124102890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff1140099c0 tx=0x7ff114009cd0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 -- 172.21.15.12:0/4067161044 shutdown_connections 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 --2- 172.21.15.12:0/4067161044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124107590 0x7ff12410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 --2- 172.21.15.12:0/4067161044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff124102890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 --2- 172.21.15.12:0/4067161044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff124102e60 0x7ff124106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 -- 172.21.15.12:0/4067161044 >> 172.21.15.12:0/4067161044 conn(0x7ff1240fdd40 msgr2=0x7ff124100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 -- 172.21.15.12:0/4067161044 shutdown_connections 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.097+0000 7ff123200700 1 -- 172.21.15.12:0/4067161044 wait complete. 2024-04-26T22:00:48.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.098+0000 7ff123200700 1 Processor -- start 2024-04-26T22:00:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.098+0000 7ff123200700 1 -- start start 2024-04-26T22:00:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff1241a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124102e60 0x7ff1241a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff124107590 0x7ff12419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff124112490 con 0x7ff124102e60 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff124112310 con 0x7ff124107590 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff123200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff124112610 con 0x7ff1241024b0 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff121e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff1241a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff121e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff1241a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:51854/0 (socket says 172.21.15.12:51854) 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff121e00700 1 -- 172.21.15.12:0/1678195032 learned_addr learned my addr 172.21.15.12:0/1678195032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff121400700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124102e60 0x7ff1241a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.099+0000 7ff122800700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff124107590 0x7ff12419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 -- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff124107590 msgr2=0x7ff12419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff124107590 0x7ff12419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 -- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124102e60 msgr2=0x7ff1241a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124102e60 0x7ff1241a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12419c250 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121400700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff124102e60 0x7ff1241a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff121e00700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff1241a1930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1140098e0 tx=0x7ff114005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114026070 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff114003c70 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400fad0 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12419c4e0 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.100+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a8800 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.101+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff124063b30 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.105+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff114029070 con 0x7ff1241024b0 2024-04-26T22:00:48.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff11aa00700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff1080a3d50 0x7ff1080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff1140d1940 con 0x7ff1241024b0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff121400700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff1080a3d50 0x7ff1080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff11aa00700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] conn(0x7ff1080aeeb0 0x7ff1080b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff114023ec0 con 0x7ff1080aeeb0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.107+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff1140d1cf0 con 0x7ff1241024b0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.108+0000 7ff122800700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] conn(0x7ff1080aeeb0 0x7ff1080b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.108+0000 7ff121400700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff1080a3d50 0x7ff1080a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff124108f50 tx=0x7ff110009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.108+0000 7ff122800700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] conn(0x7ff1080aeeb0 0x7ff1080b1350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.109+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== osd.10 v2:172.21.15.77:6824/1999551212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff114023ec0 con 0x7ff1080aeeb0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.131+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 --> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff124064000 con 0x7ff1080aeeb0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff11aa00700 1 -- 172.21.15.12:0/1678195032 <== osd.10 v2:172.21.15.77:6824/1999551212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff124064000 con 0x7ff1080aeeb0 2024-04-26T22:00:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] conn(0x7ff1080aeeb0 msgr2=0x7ff1080b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] conn(0x7ff1080aeeb0 0x7ff1080b1350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff1080a3d50 msgr2=0x7ff1080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff1080a3d50 0x7ff1080a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff124108f50 tx=0x7ff110009260 comp rx=0 tx=0).stop 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 msgr2=0x7ff1241a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff123200700 1 --2- 172.21.15.12:0/1678195032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1241024b0 0x7ff1241a1930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1140098e0 tx=0x7ff114005280 comp rx=0 tx=0).stop 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.132+0000 7ff122800700 1 -- 172.21.15.12:0/1678195032 reap_dead start 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.133+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 shutdown_connections 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.133+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 >> 172.21.15.12:0/1678195032 conn(0x7ff1240fdd40 msgr2=0x7ff1240fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.133+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 shutdown_connections 2024-04-26T22:00:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:47.133+0000 7ff123200700 1 -- 172.21.15.12:0/1678195032 wait complete. 2024-04-26T22:00:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 -- 172.21.15.12:0/2068191595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 msgr2=0x7fbc50110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 --2- 172.21.15.12:0/2068191595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 0x7fbc50110260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009fc0 tx=0x7fbc4800b330 comp rx=0 tx=0).stop 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 -- 172.21.15.12:0/2068191595 shutdown_connections 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 --2- 172.21.15.12:0/2068191595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 0x7fbc50110260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 --2- 172.21.15.12:0/2068191595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc501035b0 0x7fbc50103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 --2- 172.21.15.12:0/2068191595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc50103f60 0x7fbc50107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 -- 172.21.15.12:0/2068191595 >> 172.21.15.12:0/2068191595 conn(0x7fbc500feea0 msgr2=0x7fbc501012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 -- 172.21.15.12:0/2068191595 shutdown_connections 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.214+0000 7fbc55200700 1 -- 172.21.15.12:0/2068191595 wait complete. 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.215+0000 7fbc55200700 1 Processor -- start 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 -- start start 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc501035b0 0x7fbc5019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 0x7fbc5019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 0x7fbc50199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113580 con 0x7fbc50103f60 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113400 con 0x7fbc501035b0 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc55200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113700 con 0x7fbc50108690 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc47e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 0x7fbc5019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc47e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 0x7fbc5019fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53840/0 (socket says 172.21.15.12:53840) 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc47e00700 1 -- 172.21.15.12:0/2650464491 learned_addr learned my addr 172.21.15.12:0/2650464491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.216+0000 7fbc4f400700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc501035b0 0x7fbc5019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc4fe00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 0x7fbc50199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 -- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 msgr2=0x7fbc50199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc50108690 0x7fbc50199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 -- 172.21.15.12:0/2650464491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc501035b0 msgr2=0x7fbc5019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc501035b0 0x7fbc5019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc50199fe0 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.217+0000 7fbc47e00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 0x7fbc5019fbc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d950 tx=0x7fbc3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.218+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38016070 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.218+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019a2d0 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.218+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38004830 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.218+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38005390 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.218+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc50075650 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.219+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc50063bb0 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.221+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc38005530 con 0x7fbc50103f60 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4cc00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbc340a80e0 0x7fbc340aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbc380d0c80 con 0x7fbc50103f60 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4f400700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbc340a80e0 0x7fbc340aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4cc00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] conn(0x7fbc340b3240 0x7fbc340b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc38015ec0 con 0x7fbc340b3240 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.222+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbc380d1030 con 0x7fbc50103f60 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.223+0000 7fbc4fe00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] conn(0x7fbc340b3240 0x7fbc340b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.223+0000 7fbc4f400700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbc340a80e0 0x7fbc340aa5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005fd0 tx=0x7fbc40009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.223+0000 7fbc4fe00700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] conn(0x7fbc340b3240 0x7fbc340b56e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.224+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== osd.2 v2:172.21.15.154:6800/3122867729 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc38015ec0 con 0x7fbc340b3240 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.242+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 --> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc50063fb0 con 0x7fbc340b3240 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc4cc00700 1 -- 172.21.15.12:0/2650464491 <== osd.2 v2:172.21.15.154:6800/3122867729 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbc50063fb0 con 0x7fbc340b3240 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] conn(0x7fbc340b3240 msgr2=0x7fbc340b56e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] conn(0x7fbc340b3240 0x7fbc340b56e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbc340a80e0 msgr2=0x7fbc340aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbc340a80e0 0x7fbc340aa5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005fd0 tx=0x7fbc40009310 comp rx=0 tx=0).stop 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 msgr2=0x7fbc5019fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 --2- 172.21.15.12:0/2650464491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc50103f60 0x7fbc5019fbc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d950 tx=0x7fbc3800dc60 comp rx=0 tx=0).stop 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc4fe00700 1 -- 172.21.15.12:0/2650464491 reap_dead start 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 shutdown_connections 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.243+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 >> 172.21.15.12:0/2650464491 conn(0x7fbc500feea0 msgr2=0x7fbc50101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.244+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 shutdown_connections 2024-04-26T22:00:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.244+0000 7fbc55200700 1 -- 172.21.15.12:0/2650464491 wait complete. 2024-04-26T22:00:48.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:48 smithi012 ceph-mon[28184]: pgmap v354: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:00:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:48 smithi077 ceph-mon[32280]: pgmap v354: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:00:49.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:48 smithi154 ceph-mon[32578]: pgmap v354: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:00:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 -- 172.21.15.12:0/2299161145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075f70 msgr2=0x7fdaa410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 --2- 172.21.15.12:0/2299161145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075f70 0x7fdaa410f8e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fda90009990 tx=0x7fda90009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 -- 172.21.15.12:0/2299161145 shutdown_connections 2024-04-26T22:00:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 --2- 172.21.15.12:0/2299161145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa4112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 --2- 172.21.15.12:0/2299161145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4075650 0x7fdaa4075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 --2- 172.21.15.12:0/2299161145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075f70 0x7fdaa410f8e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 -- 172.21.15.12:0/2299161145 >> 172.21.15.12:0/2299161145 conn(0x7fdaa407a5d0 msgr2=0x7fdaa407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 -- 172.21.15.12:0/2299161145 shutdown_connections 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.105+0000 7fdaab400700 1 -- 172.21.15.12:0/2299161145 wait complete. 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 Processor -- start 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 -- start start 2024-04-26T22:00:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075650 0x7fdaa419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4075f70 0x7fdaa419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa41158b0 con 0x7fdaa4075650 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4115730 con 0x7fdaa410fe20 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.106+0000 7fdaab400700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4115a30 con 0x7fdaa4075f70 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa8a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075650 0x7fdaa419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa41a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60868/0 (socket says 172.21.15.12:60868) 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 -- 172.21.15.12:0/2985960916 learned_addr learned my addr 172.21.15.12:0/2985960916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa3e00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4075f70 0x7fdaa419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 -- 172.21.15.12:0/2985960916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4075f70 msgr2=0x7fdaa419c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4075f70 0x7fdaa419c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 -- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075650 msgr2=0x7fdaa419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075650 0x7fdaa419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa9400700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa41a0e30 con 0x7fdaa410fe20 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.107+0000 7fdaa8a00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa4075650 0x7fdaa419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaa9400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa41a06f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00ef90 tx=0x7fda9c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0092e0 con 0x7fdaa410fe20 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa41a10c0 con 0x7fdaa410fe20 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda9c004bb0 con 0x7fdaa410fe20 2024-04-26T22:00:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c004020 con 0x7fdaa410fe20 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.108+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41a89a0 con 0x7fdaa410fe20 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.109+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdaa4063b30 con 0x7fdaa410fe20 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.111+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda9c01b030 con 0x7fdaa410fe20 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.112+0000 7fdaa1600700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fda8c0a3d50 0x7fda8c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fda9c0d0bf0 con 0x7fdaa410fe20 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa8a00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fda8c0a3d50 0x7fda8c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa1600700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] conn(0x7fda8c0aeeb0 0x7fda8c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda9c003d70 con 0x7fda8c0aeeb0 2024-04-26T22:00:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fda9c0d0fa0 con 0x7fdaa410fe20 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa3e00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] conn(0x7fda8c0aeeb0 0x7fda8c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.113+0000 7fdaa8a00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fda8c0a3d50 0x7fda8c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda98011490 tx=0x7fda98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.114+0000 7fdaa3e00700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] conn(0x7fda8c0aeeb0 0x7fda8c0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.115+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== osd.3 v2:172.21.15.12:6810/1602812044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda9c003d70 con 0x7fda8c0aeeb0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.133+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 --> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdaa4063fb0 con 0x7fda8c0aeeb0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.134+0000 7fdaa1600700 1 -- 172.21.15.12:0/2985960916 <== osd.3 v2:172.21.15.12:6810/1602812044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdaa4063fb0 con 0x7fda8c0aeeb0 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.134+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] conn(0x7fda8c0aeeb0 msgr2=0x7fda8c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.134+0000 7fdaab400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] conn(0x7fda8c0aeeb0 0x7fda8c0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fda8c0a3d50 msgr2=0x7fda8c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fda8c0a3d50 0x7fda8c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda98011490 tx=0x7fda98009260 comp rx=0 tx=0).stop 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 msgr2=0x7fdaa41a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 --2- 172.21.15.12:0/2985960916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa410fe20 0x7fdaa41a06f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00ef90 tx=0x7fda9c00c3a0 comp rx=0 tx=0).stop 2024-04-26T22:00:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaa9400700 1 -- 172.21.15.12:0/2985960916 reap_dead start 2024-04-26T22:00:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 shutdown_connections 2024-04-26T22:00:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 >> 172.21.15.12:0/2985960916 conn(0x7fdaa407a5d0 msgr2=0x7fdaa41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 shutdown_connections 2024-04-26T22:00:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:48.135+0000 7fdaab400700 1 -- 172.21.15.12:0/2985960916 wait complete. 2024-04-26T22:00:50.150 INFO:teuthology.orchestra.run.smithi012.stdout:77309411407 2024-04-26T22:00:50.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T22:00:50.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 -- 172.21.15.12:0/1805560978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40760e0 msgr2=0x7f30a4113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 --2- 172.21.15.12:0/1805560978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40760e0 0x7f30a4113e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30900098f0 tx=0x7f3090009c00 comp rx=0 tx=0).stop 2024-04-26T22:00:50.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 -- 172.21.15.12:0/1805560978 shutdown_connections 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 --2- 172.21.15.12:0/1805560978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a4116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 --2- 172.21.15.12:0/1805560978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40757c0 0x7f30a4075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 --2- 172.21.15.12:0/1805560978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40760e0 0x7f30a4113e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.646+0000 7f30a8e00700 1 -- 172.21.15.12:0/1805560978 >> 172.21.15.12:0/1805560978 conn(0x7f30a40feea0 msgr2=0x7f30a41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.647+0000 7f30a8e00700 1 -- 172.21.15.12:0/1805560978 shutdown_connections 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.647+0000 7f30a8e00700 1 -- 172.21.15.12:0/1805560978 wait complete. 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.648+0000 7f30a8e00700 1 Processor -- start 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.648+0000 7f30a8e00700 1 -- start start 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 0x7f30a41a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40760e0 0x7f30a41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:50.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a41a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4119b90 con 0x7f30a40760e0 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4119a10 con 0x7f30a40757c0 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a8e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4119d10 con 0x7f30a4114360 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40760e0 0x7f30a41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a3400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 0x7f30a41a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a3e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a41a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a3e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a41a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:49252/0 (socket says 172.21.15.12:49252) 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a3e00700 1 -- 172.21.15.12:0/2556341923 learned_addr learned my addr 172.21.15.12:0/2556341923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a3400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 0x7f30a41a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:57142/0 (socket says 172.21.15.12:57142) 2024-04-26T22:00:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.649+0000 7f30a2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40760e0 0x7f30a41a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49496/0 (socket says 172.21.15.12:49496) 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 -- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 msgr2=0x7f30a41a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 0x7f30a41a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 -- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40760e0 msgr2=0x7f30a41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30a40760e0 0x7f30a41a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41a50c0 con 0x7f30a4114360 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3400700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a40757c0 0x7f30a41a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a3e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a41a4980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f309c00ef90 tx=0x7f309c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0092e0 con 0x7f30a4114360 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a5350 con 0x7f30a4114360 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c004bb0 con 0x7f30a4114360 2024-04-26T22:00:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004280 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.650+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41acc30 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.651+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30a4063b70 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.654+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f309c01b030 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.655+0000 7f309be00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f30880a3cd0 0x7f30880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.655+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f309c0d0d70 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.655+0000 7f30a3400700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f30880a3cd0 0x7f30880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.656+0000 7f309be00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] conn(0x7f30880aee30 0x7f30880b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.656+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f309c0048a0 con 0x7f30880aee30 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.656+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f309c0d1120 con 0x7f30a4114360 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.656+0000 7f30a2a00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] conn(0x7f30880aee30 0x7f30880b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.656+0000 7f30a3400700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f30880a3cd0 0x7f30880a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3094005fd0 tx=0x7f3094009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.657+0000 7f30a2a00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] conn(0x7f30880aee30 0x7f30880b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.658+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== osd.1 v2:172.21.15.77:6800/828320390 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f309c0048a0 con 0x7f30880aee30 2024-04-26T22:00:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.676+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 --> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30a4063ff0 con 0x7f30880aee30 2024-04-26T22:00:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.677+0000 7f309be00700 1 -- 172.21.15.12:0/2556341923 <== osd.1 v2:172.21.15.77:6800/828320390 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30a4063ff0 con 0x7f30880aee30 2024-04-26T22:00:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.677+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] conn(0x7f30880aee30 msgr2=0x7f30880b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.677+0000 7f30a8e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] conn(0x7f30880aee30 0x7f30880b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f30880a3cd0 msgr2=0x7f30880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f30880a3cd0 0x7f30880a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3094005fd0 tx=0x7f3094009310 comp rx=0 tx=0).stop 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 msgr2=0x7f30a41a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 --2- 172.21.15.12:0/2556341923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30a4114360 0x7f30a41a4980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f309c00ef90 tx=0x7f309c00c3a0 comp rx=0 tx=0).stop 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a3e00700 1 -- 172.21.15.12:0/2556341923 reap_dead start 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 shutdown_connections 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 >> 172.21.15.12:0/2556341923 conn(0x7f30a40feea0 msgr2=0x7f30a41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 shutdown_connections 2024-04-26T22:00:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.678+0000 7f30a8e00700 1 -- 172.21.15.12:0/2556341923 wait complete. 2024-04-26T22:00:50.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:50 smithi077 ceph-mon[32280]: pgmap v355: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:51.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:50 smithi012 ceph-mon[28184]: pgmap v355: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:51.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:50 smithi154 ceph-mon[32578]: pgmap v355: 129 pgs: 1 activating, 128 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:51.264 INFO:teuthology.orchestra.run.smithi012.stdout:107374182471 2024-04-26T22:00:51.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 -- 172.21.15.12:0/2667737218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6410fe20 msgr2=0x7f5a64112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 --2- 172.21.15.12:0/2667737218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6410fe20 0x7f5a64112210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b0a0 tx=0x7f5a5c00b3b0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 -- 172.21.15.12:0/2667737218 shutdown_connections 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 --2- 172.21.15.12:0/2667737218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6410fe20 0x7f5a64112210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 --2- 172.21.15.12:0/2667737218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a64075690 0x7f5a64075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 --2- 172.21.15.12:0/2667737218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a64075fb0 0x7f5a6410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 -- 172.21.15.12:0/2667737218 >> 172.21.15.12:0/2667737218 conn(0x7f5a6407a610 msgr2=0x7f5a6407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 -- 172.21.15.12:0/2667737218 shutdown_connections 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.379+0000 7f5a6c000700 1 -- 172.21.15.12:0/2667737218 wait complete. 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.380+0000 7f5a6c000700 1 Processor -- start 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.380+0000 7f5a6c000700 1 -- start start 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.380+0000 7f5a6c000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 0x7f5a6419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.380+0000 7f5a6c000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a64075fb0 0x7f5a6419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6c000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a6410fe20 0x7f5a641a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6c000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641158a0 con 0x7f5a64075690 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6c000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64115720 con 0x7f5a6410fe20 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6c000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64115a20 con 0x7f5a64075fb0 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 0x7f5a6419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 0x7f5a6419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49514/0 (socket says 172.21.15.12:49514) 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 -- 172.21.15.12:0/1965046809 learned_addr learned my addr 172.21.15.12:0/1965046809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6a000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a6410fe20 0x7f5a641a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a68c00700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a64075fb0 0x7f5a6419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 -- 172.21.15.12:0/1965046809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a64075fb0 msgr2=0x7f5a6419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a64075fb0 0x7f5a6419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 -- 172.21.15.12:0/1965046809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a6410fe20 msgr2=0x7f5a641a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a6410fe20 0x7f5a641a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a69600700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a641a0da0 con 0x7f5a64075690 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.381+0000 7f5a6a000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a6410fe20 0x7f5a641a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a69600700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 0x7f5a6419bd90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800eb30 tx=0x7f5a5800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800cbd0 con 0x7f5a64075690 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a580044d0 con 0x7f5a64075690 2024-04-26T22:00:51.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a641a1090 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58003e30 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.382+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a641a8910 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.383+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a64063b30 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.385+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5a58004000 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a62000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5a4c0a3d50 0x7f5a4c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5a580d10a0 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a68c00700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5a4c0a3d50 0x7f5a4c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a62000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] conn(0x7f5a4c0aeeb0 0x7f5a4c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a58018e50 con 0x7f5a4c0aeeb0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.387+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f5a580d1450 con 0x7f5a64075690 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.388+0000 7f5a6a000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] conn(0x7f5a4c0aeeb0 0x7f5a4c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.388+0000 7f5a68c00700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5a4c0a3d50 0x7f5a4c0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a64102f80 tx=0x7f5a50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.388+0000 7f5a6a000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] conn(0x7f5a4c0aeeb0 0x7f5a4c0b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.390+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== osd.0 v2:172.21.15.12:6802/1061847783 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a58018e50 con 0x7f5a4c0aeeb0 2024-04-26T22:00:51.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.409+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 --> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a64063f60 con 0x7f5a4c0aeeb0 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.409+0000 7f5a62000700 1 -- 172.21.15.12:0/1965046809 <== osd.0 v2:172.21.15.12:6802/1061847783 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5a64063f60 con 0x7f5a4c0aeeb0 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] conn(0x7f5a4c0aeeb0 msgr2=0x7f5a4c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] conn(0x7f5a4c0aeeb0 0x7f5a4c0b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5a4c0a3d50 msgr2=0x7f5a4c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f5a4c0a3d50 0x7f5a4c0a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a64102f80 tx=0x7f5a50009260 comp rx=0 tx=0).stop 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 msgr2=0x7f5a6419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 --2- 172.21.15.12:0/1965046809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a64075690 0x7f5a6419bd90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800eb30 tx=0x7f5a5800ee40 comp rx=0 tx=0).stop 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6a000700 1 -- 172.21.15.12:0/1965046809 reap_dead start 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 shutdown_connections 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 >> 172.21.15.12:0/1965046809 conn(0x7f5a6407a610 msgr2=0x7f5a64100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 shutdown_connections 2024-04-26T22:00:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:50.410+0000 7f5a6c000700 1 -- 172.21.15.12:0/1965046809 wait complete. 2024-04-26T22:00:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 -- 172.21.15.12:0/2130064736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 msgr2=0x7fa8f0075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/2130064736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f0075a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009990 tx=0x7fa8e0009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 -- 172.21.15.12:0/2130064736 shutdown_connections 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/2130064736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f0112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/2130064736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f0075a30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/2130064736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 0x7fa8f010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 -- 172.21.15.12:0/2130064736 >> 172.21.15.12:0/2130064736 conn(0x7fa8f007a580 msgr2=0x7fa8f007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 -- 172.21.15.12:0/2130064736 shutdown_connections 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.195+0000 7fa8f6e00700 1 -- 172.21.15.12:0/2130064736 wait complete. 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.196+0000 7fa8f6e00700 1 Processor -- start 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.196+0000 7fa8f6e00700 1 -- start start 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 0x7fa8f019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:52.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0115850 con 0x7fa8f010fe20 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01156d0 con 0x7fa8f0075f70 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f6e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01159d0 con 0x7fa8f0075650 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f4e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f4e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49472/0 (socket says 172.21.15.12:49472) 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8f4e00700 1 -- 172.21.15.12:0/1782196354 learned_addr learned my addr 172.21.15.12:0/1782196354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8efe00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.197+0000 7fa8e7e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 0x7fa8f019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 -- 172.21.15.12:0/1782196354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 msgr2=0x7fa8f019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 -- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 msgr2=0x7fa8f01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f01a0d50 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8efe00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f0075650 0x7fa8f019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8f4e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8f010fe20 0x7fa8f01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8e7e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 0x7fa8f019c280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e9c0 tx=0x7fa8d800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8004230 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.198+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f01a1040 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.199+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d80043d0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.199+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8003da0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.199+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f01a88c0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.199+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8f0063bb0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.202+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa8d800c7c0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.203+0000 7fa8ed600700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa8d40a3d50 0x7fa8d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.203+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa8d80d0da0 con 0x7fa8f0075f70 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.203+0000 7fa8efe00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa8d40a3d50 0x7fa8d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.204+0000 7fa8ed600700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] conn(0x7fa8d40aeeb0 0x7fa8d40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.204+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8d8011e50 con 0x7fa8d40aeeb0 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.204+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa8d80d1150 con 0x7fa8f0075f70 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.204+0000 7fa8f4e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] conn(0x7fa8d40aeeb0 0x7fa8d40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.204+0000 7fa8efe00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa8d40a3d50 0x7fa8d40a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000c00 tx=0x7fa8e000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.205+0000 7fa8f4e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] conn(0x7fa8d40aeeb0 0x7fa8d40b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.206+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== osd.9 v2:172.21.15.12:6826/1774118135 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8d8011e50 con 0x7fa8d40aeeb0 2024-04-26T22:00:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.224+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 --> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8f0064030 con 0x7fa8d40aeeb0 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.224+0000 7fa8ed600700 1 -- 172.21.15.12:0/1782196354 <== osd.9 v2:172.21.15.12:6826/1774118135 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa8f0064030 con 0x7fa8d40aeeb0 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] conn(0x7fa8d40aeeb0 msgr2=0x7fa8d40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] conn(0x7fa8d40aeeb0 0x7fa8d40b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa8d40a3d50 msgr2=0x7fa8d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fa8d40a3d50 0x7fa8d40a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0000c00 tx=0x7fa8e000b260 comp rx=0 tx=0).stop 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 msgr2=0x7fa8f019c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 --2- 172.21.15.12:0/1782196354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f0075f70 0x7fa8f019c280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e9c0 tx=0x7fa8d800ecd0 comp rx=0 tx=0).stop 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f4e00700 1 -- 172.21.15.12:0/1782196354 reap_dead start 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 shutdown_connections 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.225+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 >> 172.21.15.12:0/1782196354 conn(0x7fa8f007a580 msgr2=0x7fa8f0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.226+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 shutdown_connections 2024-04-26T22:00:52.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:49.226+0000 7fa8f6e00700 1 -- 172.21.15.12:0/1782196354 wait complete. 2024-04-26T22:00:52.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:52 smithi012 ceph-mon[28184]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:52.833 INFO:teuthology.orchestra.run.smithi012.stdout:313532612640 2024-04-26T22:00:52.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T22:00:52.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:52 smithi077 ceph-mon[32280]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:53.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 -- 172.21.15.12:0/4271946771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103470 msgr2=0x7f57241038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 --2- 172.21.15.12:0/4271946771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103470 0x7f57241038f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5710009990 tx=0x7f5710009ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 -- 172.21.15.12:0/4271946771 shutdown_connections 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 --2- 172.21.15.12:0/4271946771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5724103e30 0x7f5724110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 --2- 172.21.15.12:0/4271946771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57241094c0 0x7f57241098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 --2- 172.21.15.12:0/4271946771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103470 0x7f57241038f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 -- 172.21.15.12:0/4271946771 >> 172.21.15.12:0/4271946771 conn(0x7f572407a580 msgr2=0x7f572407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 -- 172.21.15.12:0/4271946771 shutdown_connections 2024-04-26T22:00:53.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.021+0000 7f572be00700 1 -- 172.21.15.12:0/4271946771 wait complete. 2024-04-26T22:00:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.023+0000 7f572be00700 1 Processor -- start 2024-04-26T22:00:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.023+0000 7f572be00700 1 -- start start 2024-04-26T22:00:53.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 0x7f572419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:53.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103e30 0x7f572419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:53.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 0x7f57241a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:53.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5724075f50 con 0x7f57241094c0 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5724075dd0 con 0x7f5724103e30 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f572be00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57240760d0 con 0x7f5724103470 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f5729e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 0x7f57241a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f5729400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 0x7f572419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f5729400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 0x7f572419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:49288/0 (socket says 172.21.15.12:49288) 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f5729e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 0x7f57241a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49528/0 (socket says 172.21.15.12:49528) 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.024+0000 7f5729400700 1 -- 172.21.15.12:0/2173789413 learned_addr learned my addr 172.21.15.12:0/2173789413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:53.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103e30 0x7f572419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:53.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 -- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 msgr2=0x7f572419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 0x7f572419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 -- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 msgr2=0x7f57241a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 0x7f57241a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241a0cd0 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5729e00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f57241094c0 0x7f57241a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5729400700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5724103470 0x7f572419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.025+0000 7f5728a00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103e30 0x7f572419c4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5710006010 tx=0x7f57100057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.026+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710027070 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.026+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5710022470 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.026+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a0f60 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.026+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710017650 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.026+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a8880 con 0x7f5724103e30 2024-04-26T22:00:53.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.027+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5724108320 con 0x7f5724103e30 2024-04-26T22:00:53.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.031+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f571002a070 con 0x7f5724103e30 2024-04-26T22:00:53.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.032+0000 7f5722000700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f570c0a3d50 0x7f570c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:53.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.032+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f57100d1710 con 0x7f5724103e30 2024-04-26T22:00:53.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.033+0000 7f5729400700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f570c0a3d50 0x7f570c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:53.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.033+0000 7f5722000700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] conn(0x7f570c0aeea0 0x7f570c0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:53.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.033+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f570c0b1a10 con 0x7f570c0aeea0 2024-04-26T22:00:53.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.033+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f57100d1ac0 con 0x7f5724103e30 2024-04-26T22:00:53.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.033+0000 7f5729e00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] conn(0x7f570c0aeea0 0x7f570c0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:53.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.034+0000 7f5729400700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f570c0a3d50 0x7f570c0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5724110780 tx=0x7f5718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:53.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.034+0000 7f5729e00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] conn(0x7f570c0aeea0 0x7f570c0b1340 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:53.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.035+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== osd.5 v2:172.21.15.154:6808/1747631146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f570c0b1a10 con 0x7f570c0aeea0 2024-04-26T22:00:53.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.075+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 --> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5724104b60 con 0x7f570c0aeea0 2024-04-26T22:00:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.076+0000 7f5722000700 1 -- 172.21.15.12:0/2173789413 <== osd.5 v2:172.21.15.154:6808/1747631146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5724104b60 con 0x7f570c0aeea0 2024-04-26T22:00:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.076+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] conn(0x7f570c0aeea0 msgr2=0x7f570c0b1340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.076+0000 7f572be00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] conn(0x7f570c0aeea0 0x7f570c0b1340 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f570c0a3d50 msgr2=0x7f570c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f570c0a3d50 0x7f570c0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5724110780 tx=0x7f5718009260 comp rx=0 tx=0).stop 2024-04-26T22:00:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103e30 msgr2=0x7f572419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:53.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 --2- 172.21.15.12:0/2173789413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5724103e30 0x7f572419c4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5710006010 tx=0x7f57100057c0 comp rx=0 tx=0).stop 2024-04-26T22:00:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f5729e00700 1 -- 172.21.15.12:0/2173789413 reap_dead start 2024-04-26T22:00:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 shutdown_connections 2024-04-26T22:00:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 >> 172.21.15.12:0/2173789413 conn(0x7f572407a580 msgr2=0x7f57241014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 shutdown_connections 2024-04-26T22:00:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:53.077+0000 7f572be00700 1 -- 172.21.15.12:0/2173789413 wait complete. 2024-04-26T22:00:53.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:52 smithi154 ceph-mon[32578]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 101 B/s, 0 objects/s recovering 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 -- 172.21.15.12:0/3774834948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075650 msgr2=0x7f4ad8075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 --2- 172.21.15.12:0/3774834948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075650 0x7f4ad8075a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009f90 tx=0x7f4ac800b330 comp rx=0 tx=0).stop 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 -- 172.21.15.12:0/3774834948 shutdown_connections 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 --2- 172.21.15.12:0/3774834948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad8112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 --2- 172.21.15.12:0/3774834948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075650 0x7f4ad8075a30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 --2- 172.21.15.12:0/3774834948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075f70 0x7f4ad810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 -- 172.21.15.12:0/3774834948 >> 172.21.15.12:0/3774834948 conn(0x7f4ad807a580 msgr2=0x7f4ad807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 -- 172.21.15.12:0/3774834948 shutdown_connections 2024-04-26T22:00:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.020+0000 7f4add600700 1 -- 172.21.15.12:0/3774834948 wait complete. 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.021+0000 7f4add600700 1 Processor -- start 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 -- start start 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075650 0x7f4ad819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075f70 0x7f4ad819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8115820 con 0x7f4ad810fe20 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81156a0 con 0x7f4ad8075f70 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.022+0000 7f4add600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad81159a0 con 0x7f4ad8075650 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad7400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49544/0 (socket says 172.21.15.12:49544) 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad7400700 1 -- 172.21.15.12:0/3587868696 learned_addr learned my addr 172.21.15.12:0/3587868696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075650 0x7f4ad819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ace000700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075f70 0x7f4ad819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 -- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075f70 msgr2=0x7f4ad819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ad8075f70 0x7f4ad819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 -- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 msgr2=0x7f4ad81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad6a00700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad81a0de0 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.023+0000 7f4ad7400700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ad810fe20 0x7f4ad81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4ad6a00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075650 0x7f4ad819bdd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f80 tx=0x7f4ac800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8021070 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad81a1070 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac8007ac0 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8005040 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.024+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad81a8950 con 0x7f4ad8075650 2024-04-26T22:00:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.025+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ad8063bb0 con 0x7f4ad8075650 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.027+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ac8024070 con 0x7f4ad8075650 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4acfe00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4abc0a3cd0 0x7f4abc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4ac800f020 con 0x7f4ad8075650 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4ace000700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4abc0a3cd0 0x7f4abc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4acfe00700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] conn(0x7f4abc0aed00 0x7f4abc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ac8004860 con 0x7f4abc0aed00 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.029+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4ac80d2250 con 0x7f4ad8075650 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.030+0000 7f4ad7400700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] conn(0x7f4abc0aed00 0x7f4abc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.030+0000 7f4ace000700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4abc0a3cd0 0x7f4abc0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8111b20 tx=0x7f4ac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.031+0000 7f4ad7400700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] conn(0x7f4abc0aed00 0x7f4abc0b11a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.032+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== osd.4 v2:172.21.15.77:6808/482924396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac8004860 con 0x7f4abc0aed00 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.051+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 --> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ad8063fe0 con 0x7f4abc0aed00 2024-04-26T22:00:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.051+0000 7f4acfe00700 1 -- 172.21.15.12:0/3587868696 <== osd.4 v2:172.21.15.77:6808/482924396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ad8063fe0 con 0x7f4abc0aed00 2024-04-26T22:00:54.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] conn(0x7f4abc0aed00 msgr2=0x7f4abc0b11a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] conn(0x7f4abc0aed00 0x7f4abc0b11a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4abc0a3cd0 msgr2=0x7f4abc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4abc0a3cd0 0x7f4abc0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8111b20 tx=0x7f4ac0009260 comp rx=0 tx=0).stop 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075650 msgr2=0x7f4ad819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 --2- 172.21.15.12:0/3587868696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad8075650 0x7f4ad819bdd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f80 tx=0x7f4ac800ccb0 comp rx=0 tx=0).stop 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4ad7400700 1 -- 172.21.15.12:0/3587868696 reap_dead start 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 shutdown_connections 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 >> 172.21.15.12:0/3587868696 conn(0x7f4ad807a580 msgr2=0x7f4ad81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 shutdown_connections 2024-04-26T22:00:54.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:54.052+0000 7f4add600700 1 -- 172.21.15.12:0/3587868696 wait complete. 2024-04-26T22:00:54.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:54 smithi012 ceph-mon[28184]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 85 B/s, 0 objects/s recovering 2024-04-26T22:00:54.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:54 smithi077 ceph-mon[32280]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 85 B/s, 0 objects/s recovering 2024-04-26T22:00:55.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:54 smithi154 ceph-mon[32578]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 85 B/s, 0 objects/s recovering 2024-04-26T22:00:55.280 INFO:teuthology.orchestra.run.smithi012.stdout:34359738457 2024-04-26T22:00:55.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 -- 172.21.15.12:0/2060041804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e90109490 msgr2=0x7f3e90109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 --2- 172.21.15.12:0/2060041804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e90109490 0x7f3e90109870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e840098f0 tx=0x7f3e84009c00 comp rx=0 tx=0).stop 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 -- 172.21.15.12:0/2060041804 shutdown_connections 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 --2- 172.21.15.12:0/2060041804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9006c360 0x7f3e901102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 --2- 172.21.15.12:0/2060041804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e90109490 0x7f3e90109870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 --2- 172.21.15.12:0/2060041804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e9006b9c0 0x7f3e9006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 -- 172.21.15.12:0/2060041804 >> 172.21.15.12:0/2060041804 conn(0x7f3e90079290 msgr2=0x7f3e900796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 -- 172.21.15.12:0/2060041804 shutdown_connections 2024-04-26T22:00:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.524+0000 7f3e97a00700 1 -- 172.21.15.12:0/2060041804 wait complete. 2024-04-26T22:00:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.525+0000 7f3e97a00700 1 Processor -- start 2024-04-26T22:00:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 -- start start 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 0x7f3e9019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e9006c360 0x7f3e9019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e90109490 0x7f3e901a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901135d0 con 0x7f3e90109490 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90113450 con 0x7f3e9006b9c0 2024-04-26T22:00:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e97a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90113750 con 0x7f3e9006c360 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.526+0000 7f3e95000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 0x7f3e9019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 0x7f3e9019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:57226/0 (socket says 172.21.15.12:57226) 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 -- 172.21.15.12:0/3152898948 learned_addr learned my addr 172.21.15.12:0/3152898948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e8fe00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e9006c360 0x7f3e9019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e90109490 0x7f3e901a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 -- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e9006c360 msgr2=0x7f3e9019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e9006c360 0x7f3e9019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 -- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e90109490 msgr2=0x7f3e901a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e90109490 0x7f3e901a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e901a0d30 con 0x7f3e9006b9c0 2024-04-26T22:00:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e90109490 0x7f3e901a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.527+0000 7f3e95000700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 0x7f3e9019bd20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009230 tx=0x7f3e8400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.528+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84005140 con 0x7f3e9006b9c0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.528+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e901a0fc0 con 0x7f3e9006b9c0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.528+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e8400b560 con 0x7f3e9006b9c0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.528+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84003ae0 con 0x7f3e9006b9c0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.528+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901a88a0 con 0x7f3e9006b9c0 2024-04-26T22:00:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.529+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e90063bb0 con 0x7f3e9006b9c0 2024-04-26T22:00:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.531+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e84003c80 con 0x7f3e9006b9c0 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.532+0000 7f3e8d600700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3e780a3cd0 0x7f3e780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.533+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3e840d1d40 con 0x7f3e9006b9c0 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.533+0000 7f3e8fe00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3e780a3cd0 0x7f3e780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.533+0000 7f3e8d600700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] conn(0x7f3e780aed00 0x7f3e780b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.533+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e84001d30 con 0x7f3e780aed00 2024-04-26T22:00:55.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.533+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f3e840d20f0 con 0x7f3e9006b9c0 2024-04-26T22:00:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.534+0000 7f3e95a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] conn(0x7f3e780aed00 0x7f3e780b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.534+0000 7f3e8fe00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3e780a3cd0 0x7f3e780a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3e900799d0 tx=0x7f3e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.534+0000 7f3e95a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] conn(0x7f3e780aed00 0x7f3e780b11a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.535+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== osd.8 v2:172.21.15.154:6816/165436981 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e84001d30 con 0x7f3e780aed00 2024-04-26T22:00:55.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.554+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 --> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e90063fb0 con 0x7f3e780aed00 2024-04-26T22:00:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.554+0000 7f3e8d600700 1 -- 172.21.15.12:0/3152898948 <== osd.8 v2:172.21.15.154:6816/165436981 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3e90063fb0 con 0x7f3e780aed00 2024-04-26T22:00:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] conn(0x7f3e780aed00 msgr2=0x7f3e780b11a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] conn(0x7f3e780aed00 0x7f3e780b11a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3e780a3cd0 msgr2=0x7f3e780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3e780a3cd0 0x7f3e780a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3e900799d0 tx=0x7f3e7c009260 comp rx=0 tx=0).stop 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 msgr2=0x7f3e9019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 --2- 172.21.15.12:0/3152898948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e9006b9c0 0x7f3e9019bd20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009230 tx=0x7f3e8400fa70 comp rx=0 tx=0).stop 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e95a00700 1 -- 172.21.15.12:0/3152898948 reap_dead start 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 shutdown_connections 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 >> 172.21.15.12:0/3152898948 conn(0x7f3e90079290 msgr2=0x7f3e9010e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 shutdown_connections 2024-04-26T22:00:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:55.555+0000 7f3e97a00700 1 -- 172.21.15.12:0/3152898948 wait complete. 2024-04-26T22:00:56.942 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:56 smithi154 ceph-mon[32578]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T22:00:56.967 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:56 smithi012 ceph-mon[28184]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T22:00:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:56 smithi077 ceph-mon[32280]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T22:00:57.037 INFO:teuthology.orchestra.run.smithi012.stdout:292057776165 2024-04-26T22:00:57.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T22:00:57.145 INFO:teuthology.orchestra.run.smithi012.stdout:167503724606 2024-04-26T22:00:57.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T22:00:57.761 INFO:teuthology.orchestra.run.smithi012.stdout:141733920835 2024-04-26T22:00:57.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T22:00:57.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 -- 172.21.15.12:0/2954143513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 msgr2=0x7efcf01102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:57.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 --2- 172.21.15.12:0/2954143513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01102e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efce800b0a0 tx=0x7efce800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:00:57.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 -- 172.21.15.12:0/2954143513 shutdown_connections 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 --2- 172.21.15.12:0/2954143513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01102e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 --2- 172.21.15.12:0/2954143513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf01094a0 0x7efcf0109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 --2- 172.21.15.12:0/2954143513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf006bb70 0x7efcf006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 -- 172.21.15.12:0/2954143513 >> 172.21.15.12:0/2954143513 conn(0x7efcf00792f0 msgr2=0x7efcf0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 -- 172.21.15.12:0/2954143513 shutdown_connections 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.996+0000 7efcf6a00700 1 -- 172.21.15.12:0/2954143513 wait complete. 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 Processor -- start 2024-04-26T22:00:57.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 -- start start 2024-04-26T22:00:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf006bb70 0x7efcf01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:57.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf01094a0 0x7efcf019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0113620 con 0x7efcf006bb70 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01134a0 con 0x7efcf01094a0 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.998+0000 7efcf6a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01137a0 con 0x7efcf006c510 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcefe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf006bb70 0x7efcf01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcefe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf006bb70 0x7efcf01a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49576/0 (socket says 172.21.15.12:49576) 2024-04-26T22:00:57.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcefe00700 1 -- 172.21.15.12:0/1324251164 learned_addr learned my addr 172.21.15.12:0/1324251164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcf4a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf01094a0 0x7efcf019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 -- 172.21.15.12:0/1324251164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf01094a0 msgr2=0x7efcf019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf01094a0 0x7efcf019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 -- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf006bb70 msgr2=0x7efcf01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcf006bb70 0x7efcf01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:57.999+0000 7efcef400700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf019c400 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcef400700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01a1f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009f00 tx=0x7efcdc00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00ee80 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc0044d0 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00caf0 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf019c690 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.000+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf010a100 con 0x7efcf006c510 2024-04-26T22:00:58.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.001+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcf010f870 con 0x7efcf006c510 2024-04-26T22:00:58.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.004+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efcdc026020 con 0x7efcf006c510 2024-04-26T22:00:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.005+0000 7efcecc00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efcd40a3d50 0x7efcd40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efcdc0d8e40 con 0x7efcf006c510 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcefe00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efcd40a3d50 0x7efcd40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcecc00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] conn(0x7efcd40aeeb0 0x7efcd40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcd40b1a20 con 0x7efcd40aeeb0 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7efcdc0d91f0 con 0x7efcf006c510 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.006+0000 7efcf4a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] conn(0x7efcd40aeeb0 0x7efcd40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.007+0000 7efcefe00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efcd40a3d50 0x7efcd40a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efce0009ca0 tx=0x7efce0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.007+0000 7efcf4a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] conn(0x7efcd40aeeb0 0x7efcd40b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.008+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== osd.6 v2:172.21.15.12:6818/3229053639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcd40b1a20 con 0x7efcd40aeeb0 2024-04-26T22:00:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.027+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 --> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efcf0104b50 con 0x7efcd40aeeb0 2024-04-26T22:00:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.028+0000 7efcecc00700 1 -- 172.21.15.12:0/1324251164 <== osd.6 v2:172.21.15.12:6818/3229053639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efcf0104b50 con 0x7efcd40aeeb0 2024-04-26T22:00:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.028+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] conn(0x7efcd40aeeb0 msgr2=0x7efcd40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.028+0000 7efcf6a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] conn(0x7efcd40aeeb0 0x7efcd40b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efcd40a3d50 msgr2=0x7efcd40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7efcd40a3d50 0x7efcd40a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efce0009ca0 tx=0x7efce0009260 comp rx=0 tx=0).stop 2024-04-26T22:00:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 msgr2=0x7efcf01a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 --2- 172.21.15.12:0/1324251164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf006c510 0x7efcf01a1f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009f00 tx=0x7efcdc00eca0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf4a00700 1 -- 172.21.15.12:0/1324251164 reap_dead start 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 shutdown_connections 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 >> 172.21.15.12:0/1324251164 conn(0x7efcf00792f0 msgr2=0x7efcf0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 shutdown_connections 2024-04-26T22:00:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.029+0000 7efcf6a00700 1 -- 172.21.15.12:0/1324251164 wait complete. 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 -- 172.21.15.12:0/1480529356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae4105640 msgr2=0x7f6ae4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 --2- 172.21.15.12:0/1480529356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae4105640 0x7f6ae4105ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00098f0 tx=0x7f6ad0009c00 comp rx=0 tx=0).stop 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 -- 172.21.15.12:0/1480529356 shutdown_connections 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 --2- 172.21.15.12:0/1480529356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4106000 0x7f6ae4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 --2- 172.21.15.12:0/1480529356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae410b640 0x7f6ae410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 --2- 172.21.15.12:0/1480529356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae4105640 0x7f6ae4105ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 -- 172.21.15.12:0/1480529356 >> 172.21.15.12:0/1480529356 conn(0x7f6ae4101130 msgr2=0x7f6ae4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.795+0000 7f6aeba00700 1 -- 172.21.15.12:0/1480529356 shutdown_connections 2024-04-26T22:00:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.796+0000 7f6aeba00700 1 -- 172.21.15.12:0/1480529356 wait complete. 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 Processor -- start 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 -- start start 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4105640 0x7f6ae419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae4106000 0x7f6ae419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae410b640 0x7f6ae41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4115840 con 0x7f6ae4106000 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41156c0 con 0x7f6ae410b640 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.797+0000 7f6aeba00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41159c0 con 0x7f6ae4105640 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae4106000 0x7f6ae419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4105640 0x7f6ae419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae3e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae4106000 0x7f6ae419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49592/0 (socket says 172.21.15.12:49592) 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae3e00700 1 -- 172.21.15.12:0/3276959380 learned_addr learned my addr 172.21.15.12:0/3276959380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9a00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae410b640 0x7f6ae41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 -- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae410b640 msgr2=0x7f6ae41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae410b640 0x7f6ae41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 -- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae4106000 msgr2=0x7f6ae419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae4106000 0x7f6ae419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae41a0d20 con 0x7f6ae4105640 2024-04-26T22:00:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9a00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ae410b640 0x7f6ae41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.798+0000 7f6ae9000700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4105640 0x7f6ae419bd10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e9f0 tx=0x7f6ad800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.799+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800cbd0 con 0x7f6ae4105640 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.799+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad800cd70 con 0x7f6ae4105640 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.799+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae41a1010 con 0x7f6ae4105640 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.799+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8018680 con 0x7f6ae4105640 2024-04-26T22:00:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.799+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a8890 con 0x7f6ae4105640 2024-04-26T22:00:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.800+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ae4063bb0 con 0x7f6ae4105640 2024-04-26T22:00:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.802+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ad8010430 con 0x7f6ae4105640 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.803+0000 7f6ae1600700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6acc0a3d50 0x7f6acc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6ad80d09d0 con 0x7f6ae4105640 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae3e00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6acc0a3d50 0x7f6acc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae1600700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] conn(0x7f6acc0aeeb0 0x7f6acc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ad801ce70 con 0x7f6acc0aeeb0 2024-04-26T22:00:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6ad80d0d80 con 0x7f6ae4105640 2024-04-26T22:00:58.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.804+0000 7f6ae9a00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] conn(0x7f6acc0aeeb0 0x7f6acc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:58.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.805+0000 7f6ae3e00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6acc0a3d50 0x7f6acc0a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009680 tx=0x7f6ad0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.805+0000 7f6ae9a00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] conn(0x7f6acc0aeeb0 0x7f6acc0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:58.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.806+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== osd.7 v2:172.21.15.77:6816/1953584019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ad801ce70 con 0x7f6acc0aeeb0 2024-04-26T22:00:58.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.824+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 --> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ae4063fe0 con 0x7f6acc0aeeb0 2024-04-26T22:00:58.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.825+0000 7f6ae1600700 1 -- 172.21.15.12:0/3276959380 <== osd.7 v2:172.21.15.77:6816/1953584019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ae4063fe0 con 0x7f6acc0aeeb0 2024-04-26T22:00:58.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.825+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] conn(0x7f6acc0aeeb0 msgr2=0x7f6acc0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.825+0000 7f6aeba00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] conn(0x7f6acc0aeeb0 0x7f6acc0b1350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6acc0a3d50 msgr2=0x7f6acc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6acc0a3d50 0x7f6acc0a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009680 tx=0x7f6ad0019040 comp rx=0 tx=0).stop 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4105640 msgr2=0x7f6ae419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 --2- 172.21.15.12:0/3276959380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ae4105640 0x7f6ae419bd10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800e9f0 tx=0x7f6ad800ed00 comp rx=0 tx=0).stop 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6ae9a00700 1 -- 172.21.15.12:0/3276959380 reap_dead start 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 shutdown_connections 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 >> 172.21.15.12:0/3276959380 conn(0x7f6ae4101130 msgr2=0x7f6ae4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:58.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 shutdown_connections 2024-04-26T22:00:58.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:58.826+0000 7f6aeba00700 1 -- 172.21.15.12:0/3276959380 wait complete. 2024-04-26T22:00:58.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:00:58 smithi012 ceph-mon[28184]: pgmap v359: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:00:58.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:00:58 smithi077 ceph-mon[32280]: pgmap v359: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:00:59.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:00:58 smithi154 ceph-mon[32578]: pgmap v359: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:00:59.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 -- 172.21.15.12:0/734000921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 msgr2=0x7f63a0112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 --2- 172.21.15.12:0/734000921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 0x7f63a0112210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f639800b0a0 tx=0x7f639800b3b0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 -- 172.21.15.12:0/734000921 shutdown_connections 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 --2- 172.21.15.12:0/734000921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 0x7f63a0112210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 --2- 172.21.15.12:0/734000921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63a0075650 0x7f63a0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 --2- 172.21.15.12:0/734000921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075f70 0x7f63a010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 -- 172.21.15.12:0/734000921 >> 172.21.15.12:0/734000921 conn(0x7f63a007a580 msgr2=0x7f63a007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 -- 172.21.15.12:0/734000921 shutdown_connections 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.104+0000 7f63a5e00700 1 -- 172.21.15.12:0/734000921 wait complete. 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.105+0000 7f63a5e00700 1 Processor -- start 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.105+0000 7f63a5e00700 1 -- start start 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 0x7f63a01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:59.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63a0075f70 0x7f63a01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 0x7f63a019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0115800 con 0x7f63a010fe20 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0115680 con 0x7f63a0075f70 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f63a5e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0115980 con 0x7f63a0075650 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f639ea00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 0x7f63a01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f639ea00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 0x7f63a01a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:57008/0 (socket says 172.21.15.12:57008) 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f639ea00700 1 -- 172.21.15.12:0/1876091945 learned_addr learned my addr 172.21.15.12:0/1876091945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f639f400700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 0x7f63a019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.106+0000 7f6396000700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63a0075f70 0x7f63a01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 -- 172.21.15.12:0/1876091945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63a0075f70 msgr2=0x7f63a01a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63a0075f70 0x7f63a01a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 -- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 msgr2=0x7f63a019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63a010fe20 0x7f63a019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a019c200 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f639ea00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 0x7f63a01a18e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f639000e9a0 tx=0x7f639000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639000cbd0 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639000cd70 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a019c4f0 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.107+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390018680 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.108+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a01a87b0 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.108+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63a0063bb0 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.111+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6390010430 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.112+0000 7f63a4a00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f63840a3cd0 0x7f63840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.112+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f63900d1410 con 0x7f63a0075650 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.112+0000 7f6396000700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f63840a3cd0 0x7f63840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.113+0000 7f63a4a00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] conn(0x7f63840aee30 0x7f63840b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:59.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.113+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6390027e50 con 0x7f63840aee30 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.113+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f63900d17c0 con 0x7f63a0075650 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.113+0000 7f639f400700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] conn(0x7f63840aee30 0x7f63840b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.113+0000 7f6396000700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f63840a3cd0 0x7f63840a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6388009680 tx=0x7f6388009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.114+0000 7f639f400700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] conn(0x7f63840aee30 0x7f63840b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.115+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== osd.11 v2:172.21.15.154:6824/3735995380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6390027e50 con 0x7f63840aee30 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.134+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 --> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63a0064030 con 0x7f63840aee30 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.134+0000 7f63a4a00700 1 -- 172.21.15.12:0/1876091945 <== osd.11 v2:172.21.15.154:6824/3735995380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f63a0064030 con 0x7f63840aee30 2024-04-26T22:00:59.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] conn(0x7f63840aee30 msgr2=0x7f63840b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] conn(0x7f63840aee30 0x7f63840b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f63840a3cd0 msgr2=0x7f63840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f63840a3cd0 0x7f63840a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6388009680 tx=0x7f6388009260 comp rx=0 tx=0).stop 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 msgr2=0x7f63a01a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 --2- 172.21.15.12:0/1876091945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63a0075650 0x7f63a01a18e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f639000e9a0 tx=0x7f639000ecb0 comp rx=0 tx=0).stop 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f639f400700 1 -- 172.21.15.12:0/1876091945 reap_dead start 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 shutdown_connections 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.135+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 >> 172.21.15.12:0/1876091945 conn(0x7f63a007a580 msgr2=0x7f63a01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.136+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 shutdown_connections 2024-04-26T22:00:59.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:00:59.136+0000 7f63a5e00700 1 -- 172.21.15.12:0/1876091945 wait complete. 2024-04-26T22:01:00.422 INFO:teuthology.orchestra.run.smithi012.stdout:257698037805 2024-04-26T22:01:00.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T22:01:00.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:00 smithi077 ceph-mon[32280]: pgmap v360: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:01:00.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:00 smithi012 ceph-mon[28184]: pgmap v360: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:01:00.991 INFO:teuthology.orchestra.run.smithi012.stdout:55834574931 2024-04-26T22:01:00.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T22:01:01.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:00 smithi154 ceph-mon[32578]: pgmap v360: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 67 B/s, 0 objects/s recovering 2024-04-26T22:01:01.546 INFO:teuthology.orchestra.run.smithi012.stdout:347892351006 2024-04-26T22:01:01.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T22:01:01.927 INFO:teuthology.orchestra.run.smithi012.stdout:223338299444 2024-04-26T22:01:01.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T22:01:02.310 INFO:teuthology.orchestra.run.smithi012.stdout:201863462968 2024-04-26T22:01:02.311 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T22:01:02.436 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.587 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.590 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.597 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.708 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.717 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.849 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.872 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:02.901 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:02 smithi012 ceph-mon[28184]: pgmap v361: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:02.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:02 smithi077 ceph-mon[32280]: pgmap v361: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:03.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:02 smithi154 ceph-mon[32578]: pgmap v361: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:03.718 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:04.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:04 smithi077 ceph-mon[32280]: pgmap v362: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:04.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:04 smithi012 ceph-mon[28184]: pgmap v362: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:05.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:04 smithi154 ceph-mon[32578]: pgmap v362: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:06.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:06 smithi077 ceph-mon[32280]: pgmap v363: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:07.031 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:06 smithi012 ceph-mon[28184]: pgmap v363: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:07.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:06 smithi154 ceph-mon[32578]: pgmap v363: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:08.739 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:08 smithi012 ceph-mon[28184]: pgmap v364: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:08.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:08 smithi077 ceph-mon[32280]: pgmap v364: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:09.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:08 smithi154 ceph-mon[32578]: pgmap v364: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:10 smithi012 ceph-mon[28184]: pgmap v365: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:10.977 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:10 smithi077 ceph-mon[32280]: pgmap v365: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:11.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:10 smithi154 ceph-mon[32578]: pgmap v365: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:12 smithi077 ceph-mon[32280]: pgmap v366: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:13.050 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:12 smithi012 ceph-mon[28184]: pgmap v366: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:13.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:12 smithi154 ceph-mon[32578]: pgmap v366: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.694+0000 7f1534c00700 1 -- 172.21.15.12:0/600991073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1530109a50 msgr2=0x7f1530109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.694+0000 7f1534c00700 1 --2- 172.21.15.12:0/600991073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1530109a50 0x7f1530109ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f151c009990 tx=0x7f151c009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 -- 172.21.15.12:0/600991073 shutdown_connections 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 --2- 172.21.15.12:0/600991073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010a410 0x7f1530116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 --2- 172.21.15.12:0/600991073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f153010fa50 0x7f153010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 --2- 172.21.15.12:0/600991073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1530109a50 0x7f1530109ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 -- 172.21.15.12:0/600991073 >> 172.21.15.12:0/600991073 conn(0x7f1530076bf0 msgr2=0x7f1530108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 -- 172.21.15.12:0/600991073 shutdown_connections 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.695+0000 7f1534c00700 1 -- 172.21.15.12:0/600991073 wait complete. 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.696+0000 7f1534c00700 1 Processor -- start 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.696+0000 7f1534c00700 1 -- start start 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.696+0000 7f1534c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1530109a50 0x7f15301a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.696+0000 7f1534c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.696+0000 7f1534c00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 0x7f15301a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f1534c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1530119c90 con 0x7f153010a410 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f1534c00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1530119b10 con 0x7f153010fa50 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f1534c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1530119e10 con 0x7f1530109a50 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60086/0 (socket says 172.21.15.12:60086) 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 -- 172.21.15.12:0/3448967948 learned_addr learned my addr 172.21.15.12:0/3448967948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152de00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1530109a50 0x7f15301a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152e800700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 0x7f15301a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 -- 172.21.15.12:0/3448967948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1530109a50 msgr2=0x7f15301a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1530109a50 0x7f15301a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 -- 172.21.15.12:0/3448967948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 msgr2=0x7f15301a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 0x7f15301a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152d400700 1 -- 172.21.15.12:0/3448967948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15301a5080 con 0x7f153010a410 2024-04-26T22:01:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.697+0000 7f152e800700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 0x7f15301a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f152d400700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f151c0057f0 tx=0x7f151c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c026070 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c021410 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15301a5310 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c017650 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.698+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301acbf0 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.699+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153006a110 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.702+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f151c029070 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.704+0000 7f1526a00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 0x7f15140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.704+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f151c0d1990 con 0x7f153010a410 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.704+0000 7f152de00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 0x7f15140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.705+0000 7f152de00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 0x7f15140a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1520009be0 tx=0x7f1520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.706+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f151c030080 con 0x7f153010a410 2024-04-26T22:01:13.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.951+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f153002e490 con 0x7f153010a410 2024-04-26T22:01:13.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.952+0000 7f1526a00700 1 -- 172.21.15.12:0/3448967948 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f151c013070 con 0x7f153010a410 2024-04-26T22:01:13.953 INFO:teuthology.orchestra.run.smithi012.stdout:55834574935 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 msgr2=0x7f15140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 0x7f15140a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1520009be0 tx=0x7f1520009260 comp rx=0 tx=0).stop 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 msgr2=0x7f15301a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f151c0057f0 tx=0x7f151c00bfd0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 shutdown_connections 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f153010fa50 0x7f15301a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f15140a3cd0 0x7f15140a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1530109a50 0x7f15301a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 --2- 172.21.15.12:0/3448967948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f153010a410 0x7f15301a0600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.957+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 >> 172.21.15.12:0/3448967948 conn(0x7f1530076bf0 msgr2=0x7f15300771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.958+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 shutdown_connections 2024-04-26T22:01:13.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.958+0000 7f1534c00700 1 -- 172.21.15.12:0/3448967948 wait complete. 2024-04-26T22:01:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 -- 172.21.15.12:0/3099484828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf601094c0 msgr2=0x7fcf601098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3099484828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf601094c0 0x7fcf601098a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b0a0 tx=0x7fcf5400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 -- 172.21.15.12:0/3099484828 shutdown_connections 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3099484828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 0x7fcf60110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3099484828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf601094c0 0x7fcf601098a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3099484828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf60103470 0x7fcf601038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 -- 172.21.15.12:0/3099484828 >> 172.21.15.12:0/3099484828 conn(0x7fcf6007a580 msgr2=0x7fcf6007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 -- 172.21.15.12:0/3099484828 shutdown_connections 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.806+0000 7fcf67e00700 1 -- 172.21.15.12:0/3099484828 wait complete. 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.807+0000 7fcf67e00700 1 Processor -- start 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 -- start start 2024-04-26T22:01:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 0x7fcf60075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 0x7fcf60075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60113640 con 0x7fcf60103470 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601134c0 con 0x7fcf60103e30 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf67e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601137c0 con 0x7fcf601094c0 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf65400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 0x7fcf60075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.808+0000 7fcf65e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:51460/0 (socket says 172.21.15.12:51460) 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 -- 172.21.15.12:0/3410948146 learned_addr learned my addr 172.21.15.12:0/3410948146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf64a00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 0x7fcf60075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 -- 172.21.15.12:0/3410948146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 msgr2=0x7fcf60075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 0x7fcf60075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 -- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 msgr2=0x7fcf60075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 0x7fcf60075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 -- 172.21.15.12:0/3410948146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601ac4e0 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65400700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 0x7fcf60075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.809+0000 7fcf65e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e940 tx=0x7fcf5800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.810+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58004a30 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.810+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf580054d0 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.810+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800ccf0 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.810+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601ac770 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.810+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601acc60 con 0x7fcf601094c0 2024-04-26T22:01:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.811+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6010b930 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stdout:107374182476 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.813+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf58005020 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.814+0000 7fcf5e000700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 0x7fcf480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.814+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcf580d0b40 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.815+0000 7fcf65400700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 0x7fcf480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.815+0000 7fcf65400700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 0x7fcf480a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf540062a0 tx=0x7fcf54006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:13.817+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf5809d000 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.068+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcf6010b310 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.069+0000 7fcf5e000700 1 -- 172.21.15.12:0/3410948146 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf5809c900 con 0x7fcf601094c0 2024-04-26T22:01:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.072+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 msgr2=0x7fcf480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.072+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 0x7fcf480a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf540062a0 tx=0x7fcf54006210 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.072+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 msgr2=0x7fcf600761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.072+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e940 tx=0x7fcf5800ec50 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.072+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 shutdown_connections 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcf480a3d50 0x7fcf480a6210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf601094c0 0x7fcf600761a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf60103470 0x7fcf60075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 --2- 172.21.15.12:0/3410948146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf60103e30 0x7fcf60075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 >> 172.21.15.12:0/3410948146 conn(0x7fcf6007a580 msgr2=0x7fcf60102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 shutdown_connections 2024-04-26T22:01:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.073+0000 7fcf67e00700 1 -- 172.21.15.12:0/3410948146 wait complete. 2024-04-26T22:01:14.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:14 smithi012 ceph-mon[28184]: pgmap v367: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:14.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3448967948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:01:14.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3410948146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:01:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:14 smithi077 ceph-mon[32280]: pgmap v367: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3448967948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:01:14.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3410948146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:01:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 -- 172.21.15.12:0/2086596638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec2006c360 msgr2=0x7fec201102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 --2- 172.21.15.12:0/2086596638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec2006c360 0x7fec201102d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00b0a0 tx=0x7fec1c00b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 -- 172.21.15.12:0/2086596638 shutdown_connections 2024-04-26T22:01:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 --2- 172.21.15.12:0/2086596638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec2006c360 0x7fec201102d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 --2- 172.21.15.12:0/2086596638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec20109490 0x7fec20109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 --2- 172.21.15.12:0/2086596638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006b9c0 0x7fec2006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.969+0000 7fec28800700 1 -- 172.21.15.12:0/2086596638 >> 172.21.15.12:0/2086596638 conn(0x7fec20079290 msgr2=0x7fec200796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.970+0000 7fec28800700 1 -- 172.21.15.12:0/2086596638 shutdown_connections 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.970+0000 7fec28800700 1 -- 172.21.15.12:0/2086596638 wait complete. 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 Processor -- start 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 -- start start 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 0x7fec201a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006c360 0x7fec201a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 0x7fec2019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec201135e0 con 0x7fec2006c360 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec20113460 con 0x7fec2006b9c0 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.971+0000 7fec28800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec20113760 con 0x7fec20109490 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec25400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006c360 0x7fec201a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec25e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 0x7fec201a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 0x7fec2019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec25e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 0x7fec201a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52894/0 (socket says 172.21.15.12:52894) 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec25e00700 1 -- 172.21.15.12:0/3463855227 learned_addr learned my addr 172.21.15.12:0/3463855227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 -- 172.21.15.12:0/3463855227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 msgr2=0x7fec201a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 0x7fec201a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 -- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006c360 msgr2=0x7fec201a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006c360 0x7fec201a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.972+0000 7fec26800700 1 -- 172.21.15.12:0/3463855227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2019c2a0 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec26800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 0x7fec2019bb40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0062a0 tx=0x7fec1c004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c021070 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec1c007b80 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2019c530 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c01de60 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.973+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a8800 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.974+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2006a110 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.976+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec1c024070 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.978+0000 7fec1aa00700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 0x7fec080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.978+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fec1c00f020 con 0x7fec20109490 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.978+0000 7fec25e00700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 0x7fec080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.979+0000 7fec25e00700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 0x7fec080a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec14005950 tx=0x7fec1400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:14.982+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fec1c02b020 con 0x7fec20109490 2024-04-26T22:01:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:14 smithi154 ceph-mon[32578]: pgmap v367: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3448967948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T22:01:15.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3410948146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T22:01:15.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.217+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fec201a89d0 con 0x7fec20109490 2024-04-26T22:01:15.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.218+0000 7fec1aa00700 1 -- 172.21.15.12:0/3463855227 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fec1c0074e0 con 0x7fec20109490 2024-04-26T22:01:15.219 INFO:teuthology.orchestra.run.smithi012.stdout:77309411412 2024-04-26T22:01:15.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.221+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 msgr2=0x7fec080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 0x7fec080a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec14005950 tx=0x7fec1400b500 comp rx=0 tx=0).stop 2024-04-26T22:01:15.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 msgr2=0x7fec2019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 0x7fec2019bb40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0062a0 tx=0x7fec1c004cf0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 shutdown_connections 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fec080a3d50 0x7fec080a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec20109490 0x7fec2019bb40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec2006b9c0 0x7fec201a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 --2- 172.21.15.12:0/3463855227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec2006c360 0x7fec201a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 >> 172.21.15.12:0/3463855227 conn(0x7fec20079290 msgr2=0x7fec20100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 shutdown_connections 2024-04-26T22:01:15.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:15.222+0000 7fec28800700 1 -- 172.21.15.12:0/3463855227 wait complete. 2024-04-26T22:01:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:15 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3463855227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:01:16.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:15 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3463855227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:01:16.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:15 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3463855227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T22:01:16.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:16 smithi012 ceph-mon[28184]: pgmap v368: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:17.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:16 smithi154 ceph-mon[32578]: pgmap v368: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:17.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:16 smithi077 ceph-mon[32280]: pgmap v368: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 -- 172.21.15.12:0/2312902967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 msgr2=0x7f1814110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 --2- 172.21.15.12:0/2312902967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f1814110260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f180c009fc0 tx=0x7f180c00b330 comp rx=0 tx=0).stop 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 -- 172.21.15.12:0/2312902967 shutdown_connections 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 --2- 172.21.15.12:0/2312902967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f1814110260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 --2- 172.21.15.12:0/2312902967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18141035b0 0x7f1814103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 --2- 172.21.15.12:0/2312902967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1814103f60 0x7f1814107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 -- 172.21.15.12:0/2312902967 >> 172.21.15.12:0/2312902967 conn(0x7f18140feea0 msgr2=0x7f18141012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 -- 172.21.15.12:0/2312902967 shutdown_connections 2024-04-26T22:01:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.542+0000 7f1813e00700 1 -- 172.21.15.12:0/2312902967 wait complete. 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.543+0000 7f1813e00700 1 Processor -- start 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.543+0000 7f1813e00700 1 -- start start 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18141035b0 0x7f181419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f18141a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1814113580 con 0x7f18141035b0 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1814113400 con 0x7f1814103f60 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1813e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1814113700 con 0x7f1814108690 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1812000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1812a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18141035b0 0x7f181419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1812000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52922/0 (socket says 172.21.15.12:52922) 2024-04-26T22:01:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.544+0000 7f1812000700 1 -- 172.21.15.12:0/1210025477 learned_addr learned my addr 172.21.15.12:0/1210025477 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1813400700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f18141a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 -- 172.21.15.12:0/1210025477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 msgr2=0x7f18141a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f18141a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 -- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18141035b0 msgr2=0x7f181419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18141035b0 0x7f181419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 -- 172.21.15.12:0/1210025477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18141a0d30 con 0x7f1814103f60 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.545+0000 7f1812000700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f180000ec70 tx=0x7f180000ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.546+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180000cc90 con 0x7f1814103f60 2024-04-26T22:01:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.546+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141a1020 con 0x7f1814103f60 2024-04-26T22:01:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.546+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180000ce30 con 0x7f1814103f60 2024-04-26T22:01:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.546+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a88a0 con 0x7f1814103f60 2024-04-26T22:01:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.546+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800010430 con 0x7f1814103f60 2024-04-26T22:01:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.548+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4005330 con 0x7f1814103f60 2024-04-26T22:01:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.552+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1800004020 con 0x7f1814103f60 2024-04-26T22:01:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.554+0000 7f180b400700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 0x7f17f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.554+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f18000d09c0 con 0x7f1814103f60 2024-04-26T22:01:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.554+0000 7f1812a00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 0x7f17f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.555+0000 7f1812a00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 0x7f17f80a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1804009680 tx=0x7f1804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.558+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f180009ce80 con 0x7f1814103f60 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 -- 172.21.15.12:0/836362200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5941145f0 msgr2=0x7fd5941169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 --2- 172.21.15.12:0/836362200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5941145f0 0x7fd5941169e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 -- 172.21.15.12:0/836362200 shutdown_connections 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 --2- 172.21.15.12:0/836362200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5941145f0 0x7fd5941169e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 --2- 172.21.15.12:0/836362200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd594075650 0x7fd594075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 --2- 172.21.15.12:0/836362200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075f70 0x7fd5941140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 -- 172.21.15.12:0/836362200 >> 172.21.15.12:0/836362200 conn(0x7fd59407a580 msgr2=0x7fd59407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 -- 172.21.15.12:0/836362200 shutdown_connections 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.319+0000 7fd59a000700 1 -- 172.21.15.12:0/836362200 wait complete. 2024-04-26T22:01:17.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.320+0000 7fd59a000700 1 Processor -- start 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.320+0000 7fd59a000700 1 -- start start 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd594075f70 0x7fd5941a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 0x7fd5941a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119c10 con 0x7fd594075f70 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119a90 con 0x7fd594075650 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd59a000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119d90 con 0x7fd5941145f0 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd593400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd593400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52916/0 (socket says 172.21.15.12:52916) 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd593400700 1 -- 172.21.15.12:0/2772590967 learned_addr learned my addr 172.21.15.12:0/2772590967 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd593e00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 0x7fd5941a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.321+0000 7fd58aa00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd594075f70 0x7fd5941a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 -- 172.21.15.12:0/2772590967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 msgr2=0x7fd5941a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 0x7fd5941a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 -- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd594075f70 msgr2=0x7fd5941a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd594075f70 0x7fd5941a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 -- 172.21.15.12:0/2772590967 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5941a5170 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593e00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 0x7fd5941a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.322+0000 7fd593400700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd58400ef10 tx=0x7fd58400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.323+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400c9c0 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.323+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd584019070 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.323+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5941a5400 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.323+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584014480 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.323+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941acce0 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.324+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59406a110 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.326+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd584012050 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.328+0000 7fd590c00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 0x7fd5780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.328+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd5840d0f40 con 0x7fd594075650 2024-04-26T22:01:17.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.328+0000 7fd58aa00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 0x7fd5780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.329+0000 7fd58aa00700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 0x7fd5780a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd594110fa0 tx=0x7fd57c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.330+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd584010040 con 0x7fd594075650 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stdout:347892351009 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.581+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd5941aceb0 con 0x7fd594075650 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.582+0000 7fd590c00700 1 -- 172.21.15.12:0/2772590967 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd58409d400 con 0x7fd594075650 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 msgr2=0x7fd5780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 0x7fd5780a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd594110fa0 tx=0x7fd57c009260 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 msgr2=0x7fd5941a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd58400ef10 tx=0x7fd58400c3a0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 shutdown_connections 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd5780a3cd0 0x7fd5780a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5941145f0 0x7fd5941a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd594075650 0x7fd5941a0160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 --2- 172.21.15.12:0/2772590967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd594075f70 0x7fd5941a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.585+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 >> 172.21.15.12:0/2772590967 conn(0x7fd59407a580 msgr2=0x7fd594113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.586+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 shutdown_connections 2024-04-26T22:01:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.586+0000 7fd59a000700 1 -- 172.21.15.12:0/2772590967 wait complete. 2024-04-26T22:01:17.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.805+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f17f40058f0 con 0x7f1814103f60 2024-04-26T22:01:17.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.806+0000 7f180b400700 1 -- 172.21.15.12:0/1210025477 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f180009c780 con 0x7f1814103f60 2024-04-26T22:01:17.811 INFO:teuthology.orchestra.run.smithi012.stdout:313532612646 2024-04-26T22:01:17.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.808+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 msgr2=0x7f17f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 0x7f17f80a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1804009680 tx=0x7f1804009260 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 msgr2=0x7f181419c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f180000ec70 tx=0x7f180000ef80 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 shutdown_connections 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f17f80a3d50 0x7f17f80a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1814108690 0x7f18141a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18141035b0 0x7f181419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 --2- 172.21.15.12:0/1210025477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1814103f60 0x7f181419c260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 >> 172.21.15.12:0/1210025477 conn(0x7f18140feea0 msgr2=0x7f181410eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 shutdown_connections 2024-04-26T22:01:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:17.809+0000 7f1813e00700 1 -- 172.21.15.12:0/1210025477 wait complete. 2024-04-26T22:01:18.059 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:17 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2772590967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:01:18.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:17 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2772590967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:01:18.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:17 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2772590967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T22:01:18.967 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:18 smithi012 ceph-mon[28184]: pgmap v369: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:18.967 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:18 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1210025477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:01:19.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:18 smithi154 ceph-mon[32578]: pgmap v369: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:19.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:18 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1210025477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:01:19.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:18 smithi077 ceph-mon[32280]: pgmap v369: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:19.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:18 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1210025477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T22:01:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 -- 172.21.15.12:0/3320005168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c106000 msgr2=0x7f3d5c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 --2- 172.21.15.12:0/3320005168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c106000 0x7f3d5c112510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400b0a0 tx=0x7f3d5400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 -- 172.21.15.12:0/3320005168 shutdown_connections 2024-04-26T22:01:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 --2- 172.21.15.12:0/3320005168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c106000 0x7f3d5c112510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 --2- 172.21.15.12:0/3320005168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c10b640 0x7f3d5c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 --2- 172.21.15.12:0/3320005168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c105640 0x7f3d5c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 -- 172.21.15.12:0/3320005168 >> 172.21.15.12:0/3320005168 conn(0x7f3d5c101130 msgr2=0x7f3d5c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 -- 172.21.15.12:0/3320005168 shutdown_connections 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.606+0000 7f3d61800700 1 -- 172.21.15.12:0/3320005168 wait complete. 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.607+0000 7f3d61800700 1 Processor -- start 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.607+0000 7f3d61800700 1 -- start start 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c106000 0x7f3d5c1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c10b640 0x7f3d5c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c115890 con 0x7f3d5c105640 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c115710 con 0x7f3d5c10b640 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d61800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c115a10 con 0x7f3d5c106000 2024-04-26T22:01:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d5aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d5aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58996/0 (socket says 172.21.15.12:58996) 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d5aa00700 1 -- 172.21.15.12:0/1360203537 learned_addr learned my addr 172.21.15.12:0/1360203537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d52000700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c106000 0x7f3d5c1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.608+0000 7f3d5b400700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c10b640 0x7f3d5c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 -- 172.21.15.12:0/1360203537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c106000 msgr2=0x7f3d5c1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c106000 0x7f3d5c1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 -- 172.21.15.12:0/1360203537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c10b640 msgr2=0x7f3d5c19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c10b640 0x7f3d5c19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 -- 172.21.15.12:0/1360203537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c19c2b0 con 0x7f3d5c105640 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d5aa00700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e9c0 tx=0x7f3d4c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00cbd0 con 0x7f3d5c105640 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c00ee60 con 0x7f3d5c105640 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0055c0 con 0x7f3d5c105640 2024-04-26T22:01:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c19c5a0 con 0x7f3d5c105640 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.609+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c1a8810 con 0x7f3d5c105640 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.611+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5c06a110 con 0x7f3d5c105640 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.613+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d4c004750 con 0x7f3d5c105640 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.614+0000 7f3d53e00700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 0x7f3d400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.614+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d4c0d0aa0 con 0x7f3d5c105640 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.614+0000 7f3d52000700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 0x7f3d400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.615+0000 7f3d52000700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 0x7f3d400a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009680 tx=0x7f3d44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.618+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d4c09cf60 con 0x7f3d5c105640 2024-04-26T22:01:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.844+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3d5c02e490 con 0x7f3d5c105640 2024-04-26T22:01:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.844+0000 7f3d53e00700 1 -- 172.21.15.12:0/1360203537 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3d4c09c860 con 0x7f3d5c105640 2024-04-26T22:01:19.844 INFO:teuthology.orchestra.run.smithi012.stdout:292057776171 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 msgr2=0x7f3d400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 0x7f3d400a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009680 tx=0x7f3d44009260 comp rx=0 tx=0).stop 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 msgr2=0x7f3d5c1a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e9c0 tx=0x7f3d4c00ecd0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 shutdown_connections 2024-04-26T22:01:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d5c10b640 0x7f3d5c19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c105640 0x7f3d5c1a1940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3d400a3d50 0x7f3d400a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 --2- 172.21.15.12:0/1360203537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c106000 0x7f3d5c1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 >> 172.21.15.12:0/1360203537 conn(0x7f3d5c101130 msgr2=0x7f3d5c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 shutdown_connections 2024-04-26T22:01:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:19.847+0000 7f3d61800700 1 -- 172.21.15.12:0/1360203537 wait complete. 2024-04-26T22:01:20.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612640 got 313532612646 for osd.10 2024-04-26T22:01:20.990 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:21.039 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:20 smithi077 ceph-mon[32280]: pgmap v370: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:21.039 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:20 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1360203537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:01:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:20 smithi012 ceph-mon[28184]: pgmap v370: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:21.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:20 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1360203537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:01:21.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:20 smithi154 ceph-mon[32578]: pgmap v370: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:21.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:20 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1360203537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T22:01:22.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182476 for osd.3 2024-04-26T22:01:22.237 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:23.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:22 smithi012 ceph-mon[28184]: pgmap v371: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:23.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:22 smithi154 ceph-mon[32578]: pgmap v371: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:23.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776165 got 292057776171 for osd.9 2024-04-26T22:01:23.172 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:23.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:22 smithi077 ceph-mon[32280]: pgmap v371: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 -- 172.21.15.12:0/2320529718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 msgr2=0x7f41d81098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 --2- 172.21.15.12:0/2320529718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d81098a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009990 tx=0x7f41c8009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 -- 172.21.15.12:0/2320529718 shutdown_connections 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 --2- 172.21.15.12:0/2320529718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 0x7f41d8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 --2- 172.21.15.12:0/2320529718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d81098a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 --2- 172.21.15.12:0/2320529718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 -- 172.21.15.12:0/2320529718 >> 172.21.15.12:0/2320529718 conn(0x7f41d807a580 msgr2=0x7f41d807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 -- 172.21.15.12:0/2320529718 shutdown_connections 2024-04-26T22:01:23.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.435+0000 7f41dec00700 1 -- 172.21.15.12:0/2320529718 wait complete. 2024-04-26T22:01:23.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.436+0000 7f41dec00700 1 Processor -- start 2024-04-26T22:01:23.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.436+0000 7f41dec00700 1 -- start start 2024-04-26T22:01:23.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 0x7f41d819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8113610 con 0x7f41d81094c0 2024-04-26T22:01:23.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8113490 con 0x7f41d8103470 2024-04-26T22:01:23.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dec00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8113790 con 0x7f41d8103e30 2024-04-26T22:01:23.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41d7e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41d7e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:46724/0 (socket says 172.21.15.12:46724) 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41d7e00700 1 -- 172.21.15.12:0/3721500624 learned_addr learned my addr 172.21.15.12:0/3721500624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41d7400700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 0x7f41d819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.437+0000 7f41dcc00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 -- 172.21.15.12:0/3721500624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 msgr2=0x7f41d819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 0x7f41d819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 -- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 msgr2=0x7f41d8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 -- 172.21.15.12:0/3721500624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d819a0f0 con 0x7f41d8103470 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41dcc00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:23.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d7e00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41c8000c00 tx=0x7f41c80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8027070 con 0x7f41d8103470 2024-04-26T22:01:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d819a380 con 0x7f41d8103470 2024-04-26T22:01:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c8022470 con 0x7f41d8103470 2024-04-26T22:01:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.438+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8017650 con 0x7f41d8103470 2024-04-26T22:01:23.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.439+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81a66a0 con 0x7f41d8103470 2024-04-26T22:01:23.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.440+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d806a110 con 0x7f41d8103470 2024-04-26T22:01:23.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.442+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41c802a070 con 0x7f41d8103470 2024-04-26T22:01:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.443+0000 7f41d4c00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 0x7f41bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.444+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f41c80d1630 con 0x7f41d8103470 2024-04-26T22:01:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.444+0000 7f41d7400700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 0x7f41bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.444+0000 7f41d7400700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 0x7f41bc0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009680 tx=0x7f41c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.448+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f41c8013070 con 0x7f41d8103470 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 -- 172.21.15.12:0/939615870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 msgr2=0x7f804c10ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 --2- 172.21.15.12:0/939615870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c10ec10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8040009990 tx=0x7f8040009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 -- 172.21.15.12:0/939615870 shutdown_connections 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 --2- 172.21.15.12:0/939615870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c10a320 0x7f804c1167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 --2- 172.21.15.12:0/939615870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c10ec10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 --2- 172.21.15.12:0/939615870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c109960 0x7f804c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 -- 172.21.15.12:0/939615870 >> 172.21.15.12:0/939615870 conn(0x7f804c0791b0 msgr2=0x7f804c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 -- 172.21.15.12:0/939615870 shutdown_connections 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.958+0000 7f8051e00700 1 -- 172.21.15.12:0/939615870 wait complete. 2024-04-26T22:01:23.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.959+0000 7f8051e00700 1 Processor -- start 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 -- start start 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 0x7f804c1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c10a320 0x7f804c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f804c119b70 con 0x7f804c10e830 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f804c1199f0 con 0x7f804c10a320 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f804c119cf0 con 0x7f804c109960 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59016/0 (socket says 172.21.15.12:59016) 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8051400700 1 -- 172.21.15.12:0/934823549 learned_addr learned my addr 172.21.15.12:0/934823549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.960+0000 7f8050a00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 0x7f804c1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f804be00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c10a320 0x7f804c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 -- 172.21.15.12:0/934823549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 msgr2=0x7f804c1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 0x7f804c1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 -- 172.21.15.12:0/934823549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c10a320 msgr2=0x7f804c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c10a320 0x7f804c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 -- 172.21.15.12:0/934823549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c1a5110 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8050a00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 0x7f804c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8051400700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f804400ef30 tx=0x7f804400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.961+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440092e0 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.962+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c1a53a0 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.962+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8044004bb0 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.962+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044004020 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.962+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1acc80 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.963+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c06a110 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.965+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f804401b030 con 0x7f804c10e830 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.966+0000 7f8049600700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 0x7f80340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.966+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f80440d0d20 con 0x7f804c10e830 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.967+0000 7f8050a00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 0x7f80340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.967+0000 7f8050a00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 0x7f80340a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8040006010 tx=0x7f804000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:22.969+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8044022020 con 0x7f804c10e830 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.220+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f804c02e490 con 0x7f804c10e830 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.221+0000 7f8049600700 1 -- 172.21.15.12:0/934823549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8044010040 con 0x7f804c10e830 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stdout:34359738463 2024-04-26T22:01:23.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.224+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 msgr2=0x7f80340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.224+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 0x7f80340a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8040006010 tx=0x7f804000b260 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.224+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 msgr2=0x7f804c1a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.224+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f804400ef30 tx=0x7f804400c3a0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 shutdown_connections 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804c10e830 0x7f804c1a49d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f80340a3cd0 0x7f80340a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f804c109960 0x7f804c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 --2- 172.21.15.12:0/934823549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f804c10a320 0x7f804c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 >> 172.21.15.12:0/934823549 conn(0x7f804c0791b0 msgr2=0x7f804c0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 shutdown_connections 2024-04-26T22:01:23.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.225+0000 7f8051e00700 1 -- 172.21.15.12:0/934823549 wait complete. 2024-04-26T22:01:23.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.698+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f41d802e490 con 0x7f41d8103470 2024-04-26T22:01:23.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.698+0000 7f41d4c00700 1 -- 172.21.15.12:0/3721500624 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41c809db80 con 0x7f41d8103470 2024-04-26T22:01:23.698 INFO:teuthology.orchestra.run.smithi012.stdout:141733920840 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.701+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 msgr2=0x7f41bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.701+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 0x7f41bc0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009680 tx=0x7f41c4009260 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.701+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 msgr2=0x7f41d819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.701+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41c8000c00 tx=0x7f41c80057c0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 shutdown_connections 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41d81094c0 0x7f41d8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8103470 0x7f41d819f7d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41bc0a3d50 0x7f41bc0a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 --2- 172.21.15.12:0/3721500624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41d8103e30 0x7f41d819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 >> 172.21.15.12:0/3721500624 conn(0x7f41d807a580 msgr2=0x7f41d81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 shutdown_connections 2024-04-26T22:01:23.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:23.702+0000 7f41dec00700 1 -- 172.21.15.12:0/3721500624 wait complete. 2024-04-26T22:01:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/934823549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:01:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3721500624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:01:24.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/934823549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:01:24.143 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3721500624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:01:24.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/934823549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T22:01:24.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3721500624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T22:01:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 -- 172.21.15.12:0/403888100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8481094c0 msgr2=0x7fd8481098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 --2- 172.21.15.12:0/403888100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8481094c0 0x7fd8481098a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd838009990 tx=0x7fd838009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 -- 172.21.15.12:0/403888100 shutdown_connections 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 --2- 172.21.15.12:0/403888100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd848103e30 0x7fd848110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 --2- 172.21.15.12:0/403888100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8481094c0 0x7fd8481098a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 --2- 172.21.15.12:0/403888100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd8481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 -- 172.21.15.12:0/403888100 >> 172.21.15.12:0/403888100 conn(0x7fd84807a580 msgr2=0x7fd84807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 -- 172.21.15.12:0/403888100 shutdown_connections 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.500+0000 7fd84ec00700 1 -- 172.21.15.12:0/403888100 wait complete. 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 Processor -- start 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 -- start start 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd84819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd848103e30 0x7fd84819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481135f0 con 0x7fd848103e30 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd848113470 con 0x7fd8481094c0 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.501+0000 7fd84ec00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd848113770 con 0x7fd848103470 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd847400700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd848103e30 0x7fd84819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:46748/0 (socket says 172.21.15.12:46748) 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 -- 172.21.15.12:0/1426839322 learned_addr learned my addr 172.21.15.12:0/1426839322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd847e00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd84819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 -- 172.21.15.12:0/1426839322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 msgr2=0x7fd84819be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd84819be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 -- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd848103e30 msgr2=0x7fd84819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:24.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd848103e30 0x7fd84819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd84cc00700 1 -- 172.21.15.12:0/1426839322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481a0de0 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.502+0000 7fd847e00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd84819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd84cc00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd84000ef30 tx=0x7fd84000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8400092e0 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840004bb0 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481a1070 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840004280 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.503+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481a8950 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.504+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84806a110 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.506+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd84001b030 con 0x7fd8481094c0 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.508+0000 7fd844c00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 0x7fd82c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.508+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd8400d0dd0 con 0x7fd8481094c0 2024-04-26T22:01:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.508+0000 7fd847e00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 0x7fd82c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.509+0000 7fd847e00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 0x7fd82c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd83800b2e0 tx=0x7fd83801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:24.928 INFO:teuthology.orchestra.run.smithi012.stdout:257698037811 2024-04-26T22:01:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.510+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd840022020 con 0x7fd8481094c0 2024-04-26T22:01:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.737+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd8481a8b20 con 0x7fd8481094c0 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.737+0000 7fd844c00700 1 -- 172.21.15.12:0/1426839322 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd840010040 con 0x7fd8481094c0 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 msgr2=0x7fd82c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 0x7fd82c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd83800b2e0 tx=0x7fd83801a040 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 msgr2=0x7fd8481a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd84000ef30 tx=0x7fd84000c3a0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 shutdown_connections 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8481094c0 0x7fd8481a06a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd82c0a3cd0 0x7fd82c0a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd848103470 0x7fd84819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 --2- 172.21.15.12:0/1426839322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd848103e30 0x7fd84819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.741+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 >> 172.21.15.12:0/1426839322 conn(0x7fd84807a580 msgr2=0x7fd848101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.742+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 shutdown_connections 2024-04-26T22:01:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.742+0000 7fd84ec00700 1 -- 172.21.15.12:0/1426839322 wait complete. 2024-04-26T22:01:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:24 smithi012 ceph-mon[28184]: pgmap v372: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:25.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1426839322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:01:25.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:24 smithi154 ceph-mon[32578]: pgmap v372: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:25.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1426839322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:01:25.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:24 smithi077 ceph-mon[32280]: pgmap v372: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:25.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1426839322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T22:01:25.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574931 got 55834574935 for osd.1 2024-04-26T22:01:25.906 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:26.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:25 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2370082141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:01:26.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.042 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:25 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:25 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2370082141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:01:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:25 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:25 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2370082141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T22:01:26.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:25 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:26.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738463 for osd.0 2024-04-26T22:01:26.647 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 -- 172.21.15.12:0/170792658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 msgr2=0x7fb12810f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 --2- 172.21.15.12:0/170792658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb12810f7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb114009990 tx=0x7fb114009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 -- 172.21.15.12:0/170792658 shutdown_connections 2024-04-26T22:01:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 --2- 172.21.15.12:0/170792658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12810fd00 0x7fb1281120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 --2- 172.21.15.12:0/170792658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb128102150 0x7fb128102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 --2- 172.21.15.12:0/170792658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb12810f7c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 -- 172.21.15.12:0/170792658 >> 172.21.15.12:0/170792658 conn(0x7fb1280fddb0 msgr2=0x7fb1281001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 -- 172.21.15.12:0/170792658 shutdown_connections 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.670+0000 7fb127e00700 1 -- 172.21.15.12:0/170792658 wait complete. 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.671+0000 7fb127e00700 1 Processor -- start 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.671+0000 7fb127e00700 1 -- start start 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb1281a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb12810fd00 0x7fb12819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281157b0 con 0x7fb128102b00 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb128115630 con 0x7fb128102150 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127e00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb128115930 con 0x7fb12810fd00 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb126000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb1281a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb126a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb126a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:46782/0 (socket says 172.21.15.12:46782) 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb126a00700 1 -- 172.21.15.12:0/3861522653 learned_addr learned my addr 172.21.15.12:0/3861522653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.672+0000 7fb127400700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb12810fd00 0x7fb12819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 -- 172.21.15.12:0/3861522653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb12810fd00 msgr2=0x7fb12819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb12810fd00 0x7fb12819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 -- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 msgr2=0x7fb1281a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb1281a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 -- 172.21.15.12:0/3861522653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12819c290 con 0x7fb128102150 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb126a00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb11800d980 tx=0x7fb11800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118016070 con 0x7fb128102150 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12819c580 con 0x7fb128102150 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb118004830 con 0x7fb128102150 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118005540 con 0x7fb128102150 2024-04-26T22:01:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.673+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1281a87f0 con 0x7fb128102150 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.675+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12806a110 con 0x7fb128102150 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.677+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb118004220 con 0x7fb128102150 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.679+0000 7fb11f400700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 0x7fb10c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.679+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb1180d0ed0 con 0x7fb128102150 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.679+0000 7fb126000700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 0x7fb10c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.680+0000 7fb126000700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 0x7fb10c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb128103bd0 tx=0x7fb11400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.681+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb11801f020 con 0x7fb128102150 2024-04-26T22:01:26.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.922+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb12819cfd0 con 0x7fb128102150 2024-04-26T22:01:26.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.922+0000 7fb11f400700 1 -- 172.21.15.12:0/3861522653 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb118022350 con 0x7fb128102150 2024-04-26T22:01:26.922 INFO:teuthology.orchestra.run.smithi012.stdout:167503724612 2024-04-26T22:01:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 msgr2=0x7fb10c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 0x7fb10c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb128103bd0 tx=0x7fb11400b260 comp rx=0 tx=0).stop 2024-04-26T22:01:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 msgr2=0x7fb1281a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb11800d980 tx=0x7fb11800dc90 comp rx=0 tx=0).stop 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 shutdown_connections 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb10c0a3d50 0x7fb10c0a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb12810fd00 0x7fb12819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb128102150 0x7fb1281a1920 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 --2- 172.21.15.12:0/3861522653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb128102b00 0x7fb1281a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.925+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 >> 172.21.15.12:0/3861522653 conn(0x7fb1280fddb0 msgr2=0x7fb1281001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.926+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 shutdown_connections 2024-04-26T22:01:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:26.926+0000 7fb127e00700 1 -- 172.21.15.12:0/3861522653 wait complete. 2024-04-26T22:01:27.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:26 smithi012 ceph-mon[28184]: pgmap v373: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:27.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:26 smithi154 ceph-mon[32578]: pgmap v373: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:27.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:26 smithi077 ceph-mon[32280]: pgmap v373: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:01:27.393 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351006 got 347892351009 for osd.11 2024-04-26T22:01:27.393 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:27.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037805 got 257698037811 for osd.8 2024-04-26T22:01:27.539 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.606+0000 7f04ec600700 1 -- 172.21.15.12:0/35845385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 msgr2=0x7f04e4075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.606+0000 7f04ec600700 1 --2- 172.21.15.12:0/35845385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e4075a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009990 tx=0x7f04d8009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 -- 172.21.15.12:0/35845385 shutdown_connections 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 --2- 172.21.15.12:0/35845385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e410fe20 0x7f04e4112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 --2- 172.21.15.12:0/35845385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e4075a30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 --2- 172.21.15.12:0/35845385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e4075f70 0x7f04e410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 -- 172.21.15.12:0/35845385 >> 172.21.15.12:0/35845385 conn(0x7f04e407a5d0 msgr2=0x7f04e407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 -- 172.21.15.12:0/35845385 shutdown_connections 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.607+0000 7f04ec600700 1 -- 172.21.15.12:0/35845385 wait complete. 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 Processor -- start 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 -- start start 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e4075f70 0x7f04e419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 0x7f04e41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41158b0 con 0x7f04e4075650 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4115730 con 0x7f04e410fe20 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04ec600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4115a30 con 0x7f04e4075f70 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.608+0000 7f04e9c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59052/0 (socket says 172.21.15.12:59052) 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 -- 172.21.15.12:0/2370082141 learned_addr learned my addr 172.21.15.12:0/2370082141 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9200700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e4075f70 0x7f04e419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04ea600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 0x7f04e41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 -- 172.21.15.12:0/2370082141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e4075f70 msgr2=0x7f04e419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e4075f70 0x7f04e419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 -- 172.21.15.12:0/2370082141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 msgr2=0x7f04e41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 0x7f04e41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 -- 172.21.15.12:0/2370082141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41a0dd0 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04ea600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 0x7f04e41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.609+0000 7f04e9c00700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.610+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8027070 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.610+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d8022470 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.610+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41a1060 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.610+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8017650 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.610+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a8940 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.611+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e406a110 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.613+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04d802a070 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.615+0000 7f04e2000700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 0x7f04cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.616+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f04d80d16f0 con 0x7f04e4075650 2024-04-26T22:01:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.616+0000 7f04e9200700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 0x7f04cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.616+0000 7f04e9200700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 0x7f04cc0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04e4111a70 tx=0x7f04d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.810 INFO:teuthology.orchestra.run.smithi012.stdout:223338299449 2024-04-26T22:01:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.618+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f04d8013070 con 0x7f04e4075650 2024-04-26T22:01:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.865+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f04e402e490 con 0x7f04e4075650 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.866+0000 7f04e2000700 1 -- 172.21.15.12:0/2370082141 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f04d809dc40 con 0x7f04e4075650 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 msgr2=0x7f04cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 0x7f04cc0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04e4111a70 tx=0x7f04d0009260 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 msgr2=0x7f04e419bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d80057c0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 shutdown_connections 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04e410fe20 0x7f04e41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04e4075650 0x7f04e419bdc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f04cc0a3cd0 0x7f04cc0a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 --2- 172.21.15.12:0/2370082141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e4075f70 0x7f04e419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.869+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 >> 172.21.15.12:0/2370082141 conn(0x7f04e407a5d0 msgr2=0x7f04e4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.870+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 shutdown_connections 2024-04-26T22:01:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:24.870+0000 7f04ec600700 1 -- 172.21.15.12:0/2370082141 wait complete. 2024-04-26T22:01:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3861522653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:01:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:27 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:28.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3861522653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:01:28.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:28.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:28.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:27 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3861522653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T22:01:28.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:28.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:28.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:27 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' 2024-04-26T22:01:28.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299444 got 223338299449 for osd.7 2024-04-26T22:01:28.509 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:29.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:28 smithi012 ceph-mon[28184]: pgmap v374: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:29.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:28 smithi154 ceph-mon[32578]: pgmap v374: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:29.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:28 smithi077 ceph-mon[32280]: pgmap v374: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:29.510 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411407 got 77309411412 for osd.2 2024-04-26T22:01:29.510 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:29.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724606 got 167503724612 for osd.5 2024-04-26T22:01:29.526 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:29.880 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920835 got 141733920840 for osd.4 2024-04-26T22:01:29.880 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:31.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:30 smithi012 ceph-mon[28184]: pgmap v375: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:31.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:30 smithi154 ceph-mon[32578]: pgmap v375: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:31.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:30 smithi077 ceph-mon[32280]: pgmap v375: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:01:32.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 -- 172.21.15.12:0/3053505471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c106000 msgr2=0x7f553c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 --2- 172.21.15.12:0/3053505471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c106000 0x7f553c112510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f553400b0a0 tx=0x7f553400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 -- 172.21.15.12:0/3053505471 shutdown_connections 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 --2- 172.21.15.12:0/3053505471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c106000 0x7f553c112510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 --2- 172.21.15.12:0/3053505471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c10b640 0x7f553c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 --2- 172.21.15.12:0/3053505471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c105640 0x7f553c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.244+0000 7f5544600700 1 -- 172.21.15.12:0/3053505471 >> 172.21.15.12:0/3053505471 conn(0x7f553c101130 msgr2=0x7f553c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.245+0000 7f5544600700 1 -- 172.21.15.12:0/3053505471 shutdown_connections 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.245+0000 7f5544600700 1 -- 172.21.15.12:0/3053505471 wait complete. 2024-04-26T22:01:32.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.246+0000 7f5544600700 1 Processor -- start 2024-04-26T22:01:32.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.246+0000 7f5544600700 1 -- start start 2024-04-26T22:01:32.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.246+0000 7f5544600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.246+0000 7f5544600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 0x7f553c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5544600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c10b640 0x7f553c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5544600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1158d0 con 0x7f553c105640 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5544600700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f553c115750 con 0x7f553c106000 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5544600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f553c115a50 con 0x7f553c10b640 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 0x7f553c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50800/0 (socket says 172.21.15.12:50800) 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541c00700 1 -- 172.21.15.12:0/3671683798 learned_addr learned my addr 172.21.15.12:0/3671683798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5542600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c10b640 0x7f553c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541200700 1 -- 172.21.15.12:0/3671683798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c10b640 msgr2=0x7f553c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.247+0000 7f5541200700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c10b640 0x7f553c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5541200700 1 -- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 msgr2=0x7f553c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5541200700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5541200700 1 -- 172.21.15.12:0/3671683798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c1a0d90 con 0x7f553c106000 2024-04-26T22:01:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5541c00700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:32.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5541200700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 0x7f553c19c2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552800b450 tx=0x7f552800b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528015020 con 0x7f553c106000 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5528011ce0 con 0x7f553c106000 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a1080 con 0x7f553c106000 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528010be0 con 0x7f553c106000 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.248+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a8900 con 0x7f553c106000 2024-04-26T22:01:32.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.249+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c06a110 con 0x7f553c106000 2024-04-26T22:01:32.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.252+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5528018020 con 0x7f553c106000 2024-04-26T22:01:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.253+0000 7f553a000700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 0x7f55240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.253+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f55280cfdc0 con 0x7f553c106000 2024-04-26T22:01:32.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.253+0000 7f5541c00700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 0x7f55240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.254+0000 7f5541c00700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 0x7f55240a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5530011550 tx=0x7f5530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.258+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f552801f020 con 0x7f553c106000 2024-04-26T22:01:32.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.489+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f553c19d0e0 con 0x7f553c106000 2024-04-26T22:01:32.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.490+0000 7f553a000700 1 -- 172.21.15.12:0/3671683798 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f552809c390 con 0x7f553c106000 2024-04-26T22:01:32.490 INFO:teuthology.orchestra.run.smithi012.stdout:201863462975 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 msgr2=0x7f55240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 0x7f55240a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5530011550 tx=0x7f5530009260 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 msgr2=0x7f553c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 0x7f553c19c2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552800b450 tx=0x7f552800b760 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 shutdown_connections 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55240a3d50 0x7f55240a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f553c10b640 0x7f553c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c105640 0x7f553c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 --2- 172.21.15.12:0/3671683798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f553c106000 0x7f553c19c2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 >> 172.21.15.12:0/3671683798 conn(0x7f553c101130 msgr2=0x7f553c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:32.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 shutdown_connections 2024-04-26T22:01:32.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:32.494+0000 7f5544600700 1 -- 172.21.15.12:0/3671683798 wait complete. 2024-04-26T22:01:33.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:32 smithi012 ceph-mon[28184]: pgmap v376: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:33.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:32 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3671683798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:01:33.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:32 smithi154 ceph-mon[32578]: pgmap v376: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:33.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:32 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3671683798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:01:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:32 smithi077 ceph-mon[32280]: pgmap v376: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:33.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:32 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3671683798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T22:01:33.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462968 got 201863462975 for osd.6 2024-04-26T22:01:33.368 DEBUG:teuthology.parallel:result is None 2024-04-26T22:01:33.368 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T22:01:33.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T22:01:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:35.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:34 smithi012 ceph-mon[28184]: pgmap v377: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:35.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:34 smithi154 ceph-mon[32578]: pgmap v377: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:35.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:34 smithi077 ceph-mon[32280]: pgmap v377: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 -- 172.21.15.12:0/91614276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 msgr2=0x7f8ae406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 --2- 172.21.15.12:0/91614276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae406b590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098f0 tx=0x7f8acc009c00 comp rx=0 tx=0).stop 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 -- 172.21.15.12:0/91614276 shutdown_connections 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 --2- 172.21.15.12:0/91614276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae406bad0 0x7f8ae41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 --2- 172.21.15.12:0/91614276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae41094a0 0x7f8ae4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 --2- 172.21.15.12:0/91614276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae406b590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 -- 172.21.15.12:0/91614276 >> 172.21.15.12:0/91614276 conn(0x7f8ae40792d0 msgr2=0x7f8ae40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 -- 172.21.15.12:0/91614276 shutdown_connections 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.994+0000 7f8ae9600700 1 -- 172.21.15.12:0/91614276 wait complete. 2024-04-26T22:01:35.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 Processor -- start 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 -- start start 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 0x7f8ae419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae41094a0 0x7f8ae41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae41135b0 con 0x7f8ae406b130 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4113430 con 0x7f8ae406bad0 2024-04-26T22:01:35.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.996+0000 7f8ae9600700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4113730 con 0x7f8ae41094a0 2024-04-26T22:01:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8ae2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8ae2a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50820/0 (socket says 172.21.15.12:50820) 2024-04-26T22:01:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8ae2a00700 1 -- 172.21.15.12:0/3676009925 learned_addr learned my addr 172.21.15.12:0/3676009925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 0x7f8ae419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8ae3400700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae41094a0 0x7f8ae41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 -- 172.21.15.12:0/3676009925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae41094a0 msgr2=0x7f8ae41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae41094a0 0x7f8ae41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 -- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 msgr2=0x7f8ae419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8adbe00700 1 -- 172.21.15.12:0/3676009925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae41a0d70 con 0x7f8ae406bad0 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.997+0000 7f8ae2a00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8adbe00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 0x7f8ae419c2a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc005040 con 0x7f8ae406bad0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae41a1000 con 0x7f8ae406bad0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8acc005690 con 0x7f8ae406bad0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0038e0 con 0x7f8ae406bad0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.998+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae41a88e0 con 0x7f8ae406bad0 2024-04-26T22:01:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:35.999+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae406a110 con 0x7f8ae406bad0 2024-04-26T22:01:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.001+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8acc00b760 con 0x7f8ae406bad0 2024-04-26T22:01:36.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.003+0000 7f8adb400700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 0x7f8ac80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.003+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8acc0d1670 con 0x7f8ae406bad0 2024-04-26T22:01:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.003+0000 7f8ae2a00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 0x7f8ac80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:36.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.004+0000 7f8ae2a00700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 0x7f8ac80a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4011550 tx=0x7f8ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:36.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.007+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8acc013030 con 0x7f8ae406bad0 2024-04-26T22:01:36.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.238+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ae4063860 con 0x7f8ac80a3d50 2024-04-26T22:01:36.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.251+0000 7f8adb400700 1 -- 172.21.15.12:0/3676009925 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340276 (secure 0 0 0) 0x7f8ae4063860 con 0x7f8ac80a3d50 2024-04-26T22:01:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:36.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 msgr2=0x7f8ac80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:36.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 0x7f8ac80a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4011550 tx=0x7f8ad4009260 comp rx=0 tx=0).stop 2024-04-26T22:01:36.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 msgr2=0x7f8ae419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 0x7f8ae419c2a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc00fa00 comp rx=0 tx=0).stop 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 shutdown_connections 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8ac80a3d50 0x7f8ac80a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae41094a0 0x7f8ae41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ae406b130 0x7f8ae419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 --2- 172.21.15.12:0/3676009925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ae406bad0 0x7f8ae419c2a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 >> 172.21.15.12:0/3676009925 conn(0x7f8ae40792d0 msgr2=0x7f8ae410e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 shutdown_connections 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:36.261+0000 7f8ae9600700 1 -- 172.21.15.12:0/3676009925 wait complete. 2024-04-26T22:01:36.264 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T22:01:37.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:36 smithi012 ceph-mon[28184]: pgmap v378: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:37.115 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-26T22:01:35.169185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647076,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":321,"num_read_kb":269,"num_write":264,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181552,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379940,"kb_used_data":14072,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561884,"statfs":{"total":1151940427776,"available":1151551369216,"internally_reserved":0,"allocated":14409728,"data_stored":7980962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006152"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648136+0000","last_change":"2024-04-26T21:59:44.048850+0000","last_active":"2024-04-26T22:00:44.648136+0000","last_peered":"2024-04-26T22:00:44.648136+0000","last_clean":"2024-04-26T22:00:44.648136+0000","last_became_active":"2024-04-26T21:59:44.048521+0000","last_became_peered":"2024-04-26T21:59:44.048521+0000","last_unstale":"2024-04-26T22:00:44.648136+0000","last_undegraded":"2024-04-26T22:00:44.648136+0000","last_fullsized":"2024-04-26T22:00:44.648136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:09.031610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657383+0000","last_change":"2024-04-26T21:59:44.048900+0000","last_active":"2024-04-26T22:00:44.657383+0000","last_peered":"2024-04-26T22:00:44.657383+0000","last_clean":"2024-04-26T22:00:44.657383+0000","last_became_active":"2024-04-26T21:59:44.048475+0000","last_became_peered":"2024-04-26T21:59:44.048475+0000","last_unstale":"2024-04-26T22:00:44.657383+0000","last_undegraded":"2024-04-26T22:00:44.657383+0000","last_fullsized":"2024-04-26T22:00:44.657383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:01.724256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.660196+0000","last_change":"2024-04-26T21:59:44.045875+0000","last_active":"2024-04-26T22:00:44.660196+0000","last_peered":"2024-04-26T22:00:44.660196+0000","last_clean":"2024-04-26T22:00:44.660196+0000","last_became_active":"2024-04-26T21:59:44.045413+0000","last_became_peered":"2024-04-26T21:59:44.045413+0000","last_unstale":"2024-04-26T22:00:44.660196+0000","last_undegraded":"2024-04-26T22:00:44.660196+0000","last_fullsized":"2024-04-26T22:00:44.660196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:19.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.905348+0000","last_change":"2024-04-26T22:00:44.904174+0000","last_active":"2024-04-26T22:00:44.905348+0000","last_peered":"2024-04-26T22:00:44.905348+0000","last_clean":"2024-04-26T22:00:44.905348+0000","last_became_active":"2024-04-26T22:00:44.901768+0000","last_became_peered":"2024-04-26T22:00:44.901768+0000","last_unstale":"2024-04-26T22:00:44.905348+0000","last_undegraded":"2024-04-26T22:00:44.905348+0000","last_fullsized":"2024-04-26T22:00:44.905348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:47.989221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639596+0000","last_change":"2024-04-26T21:59:44.063199+0000","last_active":"2024-04-26T22:00:44.639596+0000","last_peered":"2024-04-26T22:00:44.639596+0000","last_clean":"2024-04-26T22:00:44.639596+0000","last_became_active":"2024-04-26T21:59:44.062845+0000","last_became_peered":"2024-04-26T21:59:44.062845+0000","last_unstale":"2024-04-26T22:00:44.639596+0000","last_undegraded":"2024-04-26T22:00:44.639596+0000","last_fullsized":"2024-04-26T22:00:44.639596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:36.247652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643050+0000","last_change":"2024-04-26T21:59:44.058069+0000","last_active":"2024-04-26T22:00:44.643050+0000","last_peered":"2024-04-26T22:00:44.643050+0000","last_clean":"2024-04-26T22:00:44.643050+0000","last_became_active":"2024-04-26T21:59:44.057726+0000","last_became_peered":"2024-04-26T21:59:44.057726+0000","last_unstale":"2024-04-26T22:00:44.643050+0000","last_undegraded":"2024-04-26T22:00:44.643050+0000","last_fullsized":"2024-04-26T22:00:44.643050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:37.887034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644275+0000","last_change":"2024-04-26T21:59:44.046887+0000","last_active":"2024-04-26T22:00:44.644275+0000","last_peered":"2024-04-26T22:00:44.644275+0000","last_clean":"2024-04-26T22:00:44.644275+0000","last_became_active":"2024-04-26T21:59:44.046507+0000","last_became_peered":"2024-04-26T21:59:44.046507+0000","last_unstale":"2024-04-26T22:00:44.644275+0000","last_undegraded":"2024-04-26T22:00:44.644275+0000","last_fullsized":"2024-04-26T22:00:44.644275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:03.312480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650414+0000","last_change":"2024-04-26T21:59:44.060437+0000","last_active":"2024-04-26T22:00:44.650414+0000","last_peered":"2024-04-26T22:00:44.650414+0000","last_clean":"2024-04-26T22:00:44.650414+0000","last_became_active":"2024-04-26T21:59:44.055341+0000","last_became_peered":"2024-04-26T21:59:44.055341+0000","last_unstale":"2024-04-26T22:00:44.650414+0000","last_undegraded":"2024-04-26T22:00:44.650414+0000","last_fullsized":"2024-04-26T22:00:44.650414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:22.452619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900928+0000","last_change":"2024-04-26T21:59:44.043537+0000","last_active":"2024-04-26T22:00:44.900928+0000","last_peered":"2024-04-26T22:00:44.900928+0000","last_clean":"2024-04-26T22:00:44.900928+0000","last_became_active":"2024-04-26T21:59:44.042227+0000","last_became_peered":"2024-04-26T21:59:44.042227+0000","last_unstale":"2024-04-26T22:00:44.900928+0000","last_undegraded":"2024-04-26T22:00:44.900928+0000","last_fullsized":"2024-04-26T22:00:44.900928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.663380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643871+0000","last_change":"2024-04-26T21:59:44.039338+0000","last_active":"2024-04-26T22:00:44.643871+0000","last_peered":"2024-04-26T22:00:44.643871+0000","last_clean":"2024-04-26T22:00:44.643871+0000","last_became_active":"2024-04-26T21:59:44.039000+0000","last_became_peered":"2024-04-26T21:59:44.039000+0000","last_unstale":"2024-04-26T22:00:44.643871+0000","last_undegraded":"2024-04-26T22:00:44.643871+0000","last_fullsized":"2024-04-26T22:00:44.643871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:20.829507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645129+0000","last_change":"2024-04-26T21:59:44.046272+0000","last_active":"2024-04-26T22:00:44.645129+0000","last_peered":"2024-04-26T22:00:44.645129+0000","last_clean":"2024-04-26T22:00:44.645129+0000","last_became_active":"2024-04-26T21:59:44.045235+0000","last_became_peered":"2024-04-26T21:59:44.045235+0000","last_unstale":"2024-04-26T22:00:44.645129+0000","last_undegraded":"2024-04-26T22:00:44.645129+0000","last_fullsized":"2024-04-26T22:00:44.645129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641634+0000","last_change":"2024-04-26T21:59:44.044698+0000","last_active":"2024-04-26T22:00:44.641634+0000","last_peered":"2024-04-26T22:00:44.641634+0000","last_clean":"2024-04-26T22:00:44.641634+0000","last_became_active":"2024-04-26T21:59:44.043394+0000","last_became_peered":"2024-04-26T21:59:44.043394+0000","last_unstale":"2024-04-26T22:00:44.641634+0000","last_undegraded":"2024-04-26T22:00:44.641634+0000","last_fullsized":"2024-04-26T22:00:44.641634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:55.254581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646805+0000","last_change":"2024-04-26T21:59:44.060896+0000","last_active":"2024-04-26T22:00:44.646805+0000","last_peered":"2024-04-26T22:00:44.646805+0000","last_clean":"2024-04-26T22:00:44.646805+0000","last_became_active":"2024-04-26T21:59:44.060010+0000","last_became_peered":"2024-04-26T21:59:44.060010+0000","last_unstale":"2024-04-26T22:00:44.646805+0000","last_undegraded":"2024-04-26T22:00:44.646805+0000","last_fullsized":"2024-04-26T22:00:44.646805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:46.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.661218+0000","last_change":"2024-04-26T21:59:44.050761+0000","last_active":"2024-04-26T22:00:44.661218+0000","last_peered":"2024-04-26T22:00:44.661218+0000","last_clean":"2024-04-26T22:00:44.661218+0000","last_became_active":"2024-04-26T21:59:44.050229+0000","last_became_peered":"2024-04-26T21:59:44.050229+0000","last_unstale":"2024-04-26T22:00:44.661218+0000","last_undegraded":"2024-04-26T22:00:44.661218+0000","last_fullsized":"2024-04-26T22:00:44.661218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:46.114433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.903210+0000","last_change":"2024-04-26T22:00:44.903210+0000","last_active":"2024-04-26T22:00:44.903210+0000","last_peered":"2024-04-26T22:00:44.903210+0000","last_clean":"2024-04-26T22:00:44.903210+0000","last_became_active":"2024-04-26T22:00:44.902517+0000","last_became_peered":"2024-04-26T22:00:44.902517+0000","last_unstale":"2024-04-26T22:00:44.903210+0000","last_undegraded":"2024-04-26T22:00:44.903210+0000","last_fullsized":"2024-04-26T22:00:44.903210+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:44.244533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900249+0000","last_change":"2024-04-26T21:59:44.049616+0000","last_active":"2024-04-26T22:00:44.900249+0000","last_peered":"2024-04-26T22:00:44.900249+0000","last_clean":"2024-04-26T22:00:44.900249+0000","last_became_active":"2024-04-26T21:59:44.049259+0000","last_became_peered":"2024-04-26T21:59:44.049259+0000","last_unstale":"2024-04-26T22:00:44.900249+0000","last_undegraded":"2024-04-26T22:00:44.900249+0000","last_fullsized":"2024-04-26T22:00:44.900249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:25.264219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648002+0000","last_change":"2024-04-26T21:59:44.048657+0000","last_active":"2024-04-26T22:00:44.648002+0000","last_peered":"2024-04-26T22:00:44.648002+0000","last_clean":"2024-04-26T22:00:44.648002+0000","last_became_active":"2024-04-26T21:59:44.048209+0000","last_became_peered":"2024-04-26T21:59:44.048209+0000","last_unstale":"2024-04-26T22:00:44.648002+0000","last_undegraded":"2024-04-26T22:00:44.648002+0000","last_fullsized":"2024-04-26T22:00:44.648002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:01.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647887+0000","last_change":"2024-04-26T21:59:44.058215+0000","last_active":"2024-04-26T22:00:44.647887+0000","last_peered":"2024-04-26T22:00:44.647887+0000","last_clean":"2024-04-26T22:00:44.647887+0000","last_became_active":"2024-04-26T21:59:44.057422+0000","last_became_peered":"2024-04-26T21:59:44.057422+0000","last_unstale":"2024-04-26T22:00:44.647887+0000","last_undegraded":"2024-04-26T22:00:44.647887+0000","last_fullsized":"2024-04-26T22:00:44.647887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:17.215168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900764+0000","last_change":"2024-04-26T21:59:44.054840+0000","last_active":"2024-04-26T22:00:44.900764+0000","last_peered":"2024-04-26T22:00:44.900764+0000","last_clean":"2024-04-26T22:00:44.900764+0000","last_became_active":"2024-04-26T21:59:44.054629+0000","last_became_peered":"2024-04-26T21:59:44.054629+0000","last_unstale":"2024-04-26T22:00:44.900764+0000","last_undegraded":"2024-04-26T22:00:44.900764+0000","last_fullsized":"2024-04-26T22:00:44.900764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:37.204066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900710+0000","last_change":"2024-04-26T21:59:48.145585+0000","last_active":"2024-04-26T22:00:44.900710+0000","last_peered":"2024-04-26T22:00:44.900710+0000","last_clean":"2024-04-26T22:00:44.900710+0000","last_became_active":"2024-04-26T21:59:48.145283+0000","last_became_peered":"2024-04-26T21:59:48.145283+0000","last_unstale":"2024-04-26T22:00:44.900710+0000","last_undegraded":"2024-04-26T22:00:44.900710+0000","last_fullsized":"2024-04-26T22:00:44.900710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:53.096333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.909389+0000","last_change":"2024-04-26T22:00:44.909389+0000","last_active":"2024-04-26T22:00:44.909389+0000","last_peered":"2024-04-26T22:00:44.909389+0000","last_clean":"2024-04-26T22:00:44.909389+0000","last_became_active":"2024-04-26T22:00:44.902131+0000","last_became_peered":"2024-04-26T22:00:44.902131+0000","last_unstale":"2024-04-26T22:00:44.909389+0000","last_undegraded":"2024-04-26T22:00:44.909389+0000","last_fullsized":"2024-04-26T22:00:44.909389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:30.526396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656363+0000","last_change":"2024-04-26T21:59:44.060192+0000","last_active":"2024-04-26T22:00:44.656363+0000","last_peered":"2024-04-26T22:00:44.656363+0000","last_clean":"2024-04-26T22:00:44.656363+0000","last_became_active":"2024-04-26T21:59:44.054969+0000","last_became_peered":"2024-04-26T21:59:44.054969+0000","last_unstale":"2024-04-26T22:00:44.656363+0000","last_undegraded":"2024-04-26T22:00:44.656363+0000","last_fullsized":"2024-04-26T22:00:44.656363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:54.670629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640118+0000","last_change":"2024-04-26T21:59:48.144947+0000","last_active":"2024-04-26T22:00:44.640118+0000","last_peered":"2024-04-26T22:00:44.640118+0000","last_clean":"2024-04-26T22:00:44.640118+0000","last_became_active":"2024-04-26T21:59:48.144547+0000","last_became_peered":"2024-04-26T21:59:48.144547+0000","last_unstale":"2024-04-26T22:00:44.640118+0000","last_undegraded":"2024-04-26T22:00:44.640118+0000","last_fullsized":"2024-04-26T22:00:44.640118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.492136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656735+0000","last_change":"2024-04-26T21:59:43.071253+0000","last_active":"2024-04-26T22:00:44.656735+0000","last_peered":"2024-04-26T22:00:44.656735+0000","last_clean":"2024-04-26T22:00:44.656735+0000","last_became_active":"2024-04-26T21:59:43.070926+0000","last_became_peered":"2024-04-26T21:59:43.070926+0000","last_unstale":"2024-04-26T22:00:44.656735+0000","last_undegraded":"2024-04-26T22:00:44.656735+0000","last_fullsized":"2024-04-26T22:00:44.656735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:13.280936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.662577+0000","last_change":"2024-04-26T21:59:44.051951+0000","last_active":"2024-04-26T22:00:44.662577+0000","last_peered":"2024-04-26T22:00:44.662577+0000","last_clean":"2024-04-26T22:00:44.662577+0000","last_became_active":"2024-04-26T21:59:44.051206+0000","last_became_peered":"2024-04-26T21:59:44.051206+0000","last_unstale":"2024-04-26T22:00:44.662577+0000","last_undegraded":"2024-04-26T22:00:44.662577+0000","last_fullsized":"2024-04-26T22:00:44.662577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:05.034106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.659464+0000","last_change":"2024-04-26T22:00:44.659464+0000","last_active":"2024-04-26T22:00:44.659464+0000","last_peered":"2024-04-26T22:00:44.659464+0000","last_clean":"2024-04-26T22:00:44.659464+0000","last_became_active":"2024-04-26T22:00:44.659008+0000","last_became_peered":"2024-04-26T22:00:44.659008+0000","last_unstale":"2024-04-26T22:00:44.659464+0000","last_undegraded":"2024-04-26T22:00:44.659464+0000","last_fullsized":"2024-04-26T22:00:44.659464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:42.042268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640094+0000","last_change":"2024-04-26T21:59:43.038322+0000","last_active":"2024-04-26T22:00:44.640094+0000","last_peered":"2024-04-26T22:00:44.640094+0000","last_clean":"2024-04-26T22:00:44.640094+0000","last_became_active":"2024-04-26T21:59:43.037968+0000","last_became_peered":"2024-04-26T21:59:43.037968+0000","last_unstale":"2024-04-26T22:00:44.640094+0000","last_undegraded":"2024-04-26T22:00:44.640094+0000","last_fullsized":"2024-04-26T22:00:44.640094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:21.265367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900539+0000","last_change":"2024-04-26T21:59:44.054970+0000","last_active":"2024-04-26T22:00:44.900539+0000","last_peered":"2024-04-26T22:00:44.900539+0000","last_clean":"2024-04-26T22:00:44.900539+0000","last_became_active":"2024-04-26T21:59:44.054599+0000","last_became_peered":"2024-04-26T21:59:44.054599+0000","last_unstale":"2024-04-26T22:00:44.900539+0000","last_undegraded":"2024-04-26T22:00:44.900539+0000","last_fullsized":"2024-04-26T22:00:44.900539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:22.048544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640401+0000","last_change":"2024-04-26T21:59:48.136704+0000","last_active":"2024-04-26T22:00:44.640401+0000","last_peered":"2024-04-26T22:00:44.640401+0000","last_clean":"2024-04-26T22:00:44.640401+0000","last_became_active":"2024-04-26T21:59:48.136276+0000","last_became_peered":"2024-04-26T21:59:48.136276+0000","last_unstale":"2024-04-26T22:00:44.640401+0000","last_undegraded":"2024-04-26T22:00:44.640401+0000","last_fullsized":"2024-04-26T22:00:44.640401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:05.608497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639812+0000","last_change":"2024-04-26T21:59:43.031556+0000","last_active":"2024-04-26T22:00:44.639812+0000","last_peered":"2024-04-26T22:00:44.639812+0000","last_clean":"2024-04-26T22:00:44.639812+0000","last_became_active":"2024-04-26T21:59:43.031216+0000","last_became_peered":"2024-04-26T21:59:43.031216+0000","last_unstale":"2024-04-26T22:00:44.639812+0000","last_undegraded":"2024-04-26T22:00:44.639812+0000","last_fullsized":"2024-04-26T22:00:44.639812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:51.675630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647111+0000","last_change":"2024-04-26T21:59:44.048291+0000","last_active":"2024-04-26T22:00:44.647111+0000","last_peered":"2024-04-26T22:00:44.647111+0000","last_clean":"2024-04-26T22:00:44.647111+0000","last_became_active":"2024-04-26T21:59:44.047983+0000","last_became_peered":"2024-04-26T21:59:44.047983+0000","last_unstale":"2024-04-26T22:00:44.647111+0000","last_undegraded":"2024-04-26T22:00:44.647111+0000","last_fullsized":"2024-04-26T22:00:44.647111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:17.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900611+0000","last_change":"2024-04-26T21:59:48.129287+0000","last_active":"2024-04-26T22:00:44.900611+0000","last_peered":"2024-04-26T22:00:44.900611+0000","last_clean":"2024-04-26T22:00:44.900611+0000","last_became_active":"2024-04-26T21:59:48.128789+0000","last_became_peered":"2024-04-26T21:59:48.128789+0000","last_unstale":"2024-04-26T22:00:44.900611+0000","last_undegraded":"2024-04-26T22:00:44.900611+0000","last_fullsized":"2024-04-26T22:00:44.900611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:46.495215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.649352+0000","last_change":"2024-04-26T22:00:44.649352+0000","last_active":"2024-04-26T22:00:44.649352+0000","last_peered":"2024-04-26T22:00:44.649352+0000","last_clean":"2024-04-26T22:00:44.649352+0000","last_became_active":"2024-04-26T22:00:44.648771+0000","last_became_peered":"2024-04-26T22:00:44.648771+0000","last_unstale":"2024-04-26T22:00:44.649352+0000","last_undegraded":"2024-04-26T22:00:44.649352+0000","last_fullsized":"2024-04-26T22:00:44.649352+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:57.718115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639606+0000","last_change":"2024-04-26T21:59:44.058077+0000","last_active":"2024-04-26T22:00:44.639606+0000","last_peered":"2024-04-26T22:00:44.639606+0000","last_clean":"2024-04-26T22:00:44.639606+0000","last_became_active":"2024-04-26T21:59:44.057720+0000","last_became_peered":"2024-04-26T21:59:44.057720+0000","last_unstale":"2024-04-26T22:00:44.639606+0000","last_undegraded":"2024-04-26T22:00:44.639606+0000","last_fullsized":"2024-04-26T22:00:44.639606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:56.889733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900405+0000","last_change":"2024-04-26T21:59:48.149256+0000","last_active":"2024-04-26T22:00:44.900405+0000","last_peered":"2024-04-26T22:00:44.900405+0000","last_clean":"2024-04-26T22:00:44.900405+0000","last_became_active":"2024-04-26T21:59:48.148832+0000","last_became_peered":"2024-04-26T21:59:48.148832+0000","last_unstale":"2024-04-26T22:00:44.900405+0000","last_undegraded":"2024-04-26T22:00:44.900405+0000","last_fullsized":"2024-04-26T22:00:44.900405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:46.120634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.642987+0000","last_change":"2024-04-26T21:59:43.050527+0000","last_active":"2024-04-26T22:00:44.642987+0000","last_peered":"2024-04-26T22:00:44.642987+0000","last_clean":"2024-04-26T22:00:44.642987+0000","last_became_active":"2024-04-26T21:59:43.050168+0000","last_became_peered":"2024-04-26T21:59:43.050168+0000","last_unstale":"2024-04-26T22:00:44.642987+0000","last_undegraded":"2024-04-26T22:00:44.642987+0000","last_fullsized":"2024-04-26T22:00:44.642987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:45.484392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647019+0000","last_change":"2024-04-26T21:59:44.063479+0000","last_active":"2024-04-26T22:00:44.647019+0000","last_peered":"2024-04-26T22:00:44.647019+0000","last_clean":"2024-04-26T22:00:44.647019+0000","last_became_active":"2024-04-26T21:59:44.063205+0000","last_became_peered":"2024-04-26T21:59:44.063205+0000","last_unstale":"2024-04-26T22:00:44.647019+0000","last_undegraded":"2024-04-26T22:00:44.647019+0000","last_fullsized":"2024-04-26T22:00:44.647019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:07.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647921+0000","last_change":"2024-04-26T21:59:48.155383+0000","last_active":"2024-04-26T22:00:44.647921+0000","last_peered":"2024-04-26T22:00:44.647921+0000","last_clean":"2024-04-26T22:00:44.647921+0000","last_became_active":"2024-04-26T21:59:48.154948+0000","last_became_peered":"2024-04-26T21:59:48.154948+0000","last_unstale":"2024-04-26T22:00:44.647921+0000","last_undegraded":"2024-04-26T22:00:44.647921+0000","last_fullsized":"2024-04-26T22:00:44.647921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:25.874999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645728+0000","last_change":"2024-04-26T21:59:43.061828+0000","last_active":"2024-04-26T22:00:44.645728+0000","last_peered":"2024-04-26T22:00:44.645728+0000","last_clean":"2024-04-26T22:00:44.645728+0000","last_became_active":"2024-04-26T21:59:43.061440+0000","last_became_peered":"2024-04-26T21:59:43.061440+0000","last_unstale":"2024-04-26T22:00:44.645728+0000","last_undegraded":"2024-04-26T22:00:44.645728+0000","last_fullsized":"2024-04-26T22:00:44.645728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:17.964410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650645+0000","last_change":"2024-04-26T21:59:44.056947+0000","last_active":"2024-04-26T22:00:44.650645+0000","last_peered":"2024-04-26T22:00:44.650645+0000","last_clean":"2024-04-26T22:00:44.650645+0000","last_became_active":"2024-04-26T21:59:44.056537+0000","last_became_peered":"2024-04-26T21:59:44.056537+0000","last_unstale":"2024-04-26T22:00:44.650645+0000","last_undegraded":"2024-04-26T22:00:44.650645+0000","last_fullsized":"2024-04-26T22:00:44.650645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:19.601355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657265+0000","last_change":"2024-04-26T21:59:48.141476+0000","last_active":"2024-04-26T22:00:44.657265+0000","last_peered":"2024-04-26T22:00:44.657265+0000","last_clean":"2024-04-26T22:00:44.657265+0000","last_became_active":"2024-04-26T21:59:48.141028+0000","last_became_peered":"2024-04-26T21:59:48.141028+0000","last_unstale":"2024-04-26T22:00:44.657265+0000","last_undegraded":"2024-04-26T22:00:44.657265+0000","last_fullsized":"2024-04-26T22:00:44.657265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:36.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646537+0000","last_change":"2024-04-26T21:59:43.035587+0000","last_active":"2024-04-26T22:00:44.646537+0000","last_peered":"2024-04-26T22:00:44.646537+0000","last_clean":"2024-04-26T22:00:44.646537+0000","last_became_active":"2024-04-26T21:59:43.035102+0000","last_became_peered":"2024-04-26T21:59:43.035102+0000","last_unstale":"2024-04-26T22:00:44.646537+0000","last_undegraded":"2024-04-26T22:00:44.646537+0000","last_fullsized":"2024-04-26T22:00:44.646537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:52.817321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648389+0000","last_change":"2024-04-26T21:59:44.062002+0000","last_active":"2024-04-26T22:00:44.648389+0000","last_peered":"2024-04-26T22:00:44.648389+0000","last_clean":"2024-04-26T22:00:44.648389+0000","last_became_active":"2024-04-26T21:59:44.058419+0000","last_became_peered":"2024-04-26T21:59:44.058419+0000","last_unstale":"2024-04-26T22:00:44.648389+0000","last_undegraded":"2024-04-26T22:00:44.648389+0000","last_fullsized":"2024-04-26T22:00:44.648389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:48.698093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650871+0000","last_change":"2024-04-26T21:59:48.134540+0000","last_active":"2024-04-26T22:00:44.650871+0000","last_peered":"2024-04-26T22:00:44.650871+0000","last_clean":"2024-04-26T22:00:44.650871+0000","last_became_active":"2024-04-26T21:59:48.133149+0000","last_became_peered":"2024-04-26T21:59:48.133149+0000","last_unstale":"2024-04-26T22:00:44.650871+0000","last_undegraded":"2024-04-26T22:00:44.650871+0000","last_fullsized":"2024-04-26T22:00:44.650871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:18.861332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643614+0000","last_change":"2024-04-26T21:59:43.053360+0000","last_active":"2024-04-26T22:00:44.643614+0000","last_peered":"2024-04-26T22:00:44.643614+0000","last_clean":"2024-04-26T22:00:44.643614+0000","last_became_active":"2024-04-26T21:59:43.053113+0000","last_became_peered":"2024-04-26T21:59:43.053113+0000","last_unstale":"2024-04-26T22:00:44.643614+0000","last_undegraded":"2024-04-26T22:00:44.643614+0000","last_fullsized":"2024-04-26T22:00:44.643614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:38.464573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900806+0000","last_change":"2024-04-26T21:59:44.060319+0000","last_active":"2024-04-26T22:00:44.900806+0000","last_peered":"2024-04-26T22:00:44.900806+0000","last_clean":"2024-04-26T22:00:44.900806+0000","last_became_active":"2024-04-26T21:59:44.059947+0000","last_became_peered":"2024-04-26T21:59:44.059947+0000","last_unstale":"2024-04-26T22:00:44.900806+0000","last_undegraded":"2024-04-26T22:00:44.900806+0000","last_fullsized":"2024-04-26T22:00:44.900806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:01.851669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644179+0000","last_change":"2024-04-26T21:59:48.148358+0000","last_active":"2024-04-26T22:00:44.644179+0000","last_peered":"2024-04-26T22:00:44.644179+0000","last_clean":"2024-04-26T22:00:44.644179+0000","last_became_active":"2024-04-26T21:59:48.143520+0000","last_became_peered":"2024-04-26T21:59:48.143520+0000","last_unstale":"2024-04-26T22:00:44.644179+0000","last_undegraded":"2024-04-26T22:00:44.644179+0000","last_fullsized":"2024-04-26T22:00:44.644179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:23.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645042+0000","last_change":"2024-04-26T21:59:43.037153+0000","last_active":"2024-04-26T22:00:44.645042+0000","last_peered":"2024-04-26T22:00:44.645042+0000","last_clean":"2024-04-26T22:00:44.645042+0000","last_became_active":"2024-04-26T21:59:43.036816+0000","last_became_peered":"2024-04-26T21:59:43.036816+0000","last_unstale":"2024-04-26T22:00:44.645042+0000","last_undegraded":"2024-04-26T22:00:44.645042+0000","last_fullsized":"2024-04-26T22:00:44.645042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:44.655304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.662701+0000","last_change":"2024-04-26T21:59:44.047947+0000","last_active":"2024-04-26T22:00:44.662701+0000","last_peered":"2024-04-26T22:00:44.662701+0000","last_clean":"2024-04-26T22:00:44.662701+0000","last_became_active":"2024-04-26T21:59:44.047586+0000","last_became_peered":"2024-04-26T21:59:44.047586+0000","last_unstale":"2024-04-26T22:00:44.662701+0000","last_undegraded":"2024-04-26T22:00:44.662701+0000","last_fullsized":"2024-04-26T22:00:44.662701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:49.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641250+0000","last_change":"2024-04-26T21:59:48.150787+0000","last_active":"2024-04-26T22:00:44.641250+0000","last_peered":"2024-04-26T22:00:44.641250+0000","last_clean":"2024-04-26T22:00:44.641250+0000","last_became_active":"2024-04-26T21:59:48.150004+0000","last_became_peered":"2024-04-26T21:59:48.150004+0000","last_unstale":"2024-04-26T22:00:44.641250+0000","last_undegraded":"2024-04-26T22:00:44.641250+0000","last_fullsized":"2024-04-26T22:00:44.641250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:19.657837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646863+0000","last_change":"2024-04-26T21:59:43.050931+0000","last_active":"2024-04-26T22:00:44.646863+0000","last_peered":"2024-04-26T22:00:44.646863+0000","last_clean":"2024-04-26T22:00:44.646863+0000","last_became_active":"2024-04-26T21:59:43.050508+0000","last_became_peered":"2024-04-26T21:59:43.050508+0000","last_unstale":"2024-04-26T22:00:44.646863+0000","last_undegraded":"2024-04-26T22:00:44.646863+0000","last_fullsized":"2024-04-26T22:00:44.646863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:08.378649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900623+0000","last_change":"2024-04-26T21:59:44.039078+0000","last_active":"2024-04-26T22:00:44.900623+0000","last_peered":"2024-04-26T22:00:44.900623+0000","last_clean":"2024-04-26T22:00:44.900623+0000","last_became_active":"2024-04-26T21:59:44.038734+0000","last_became_peered":"2024-04-26T21:59:44.038734+0000","last_unstale":"2024-04-26T22:00:44.900623+0000","last_undegraded":"2024-04-26T22:00:44.900623+0000","last_fullsized":"2024-04-26T22:00:44.900623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:07.551134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644453+0000","last_change":"2024-04-26T21:59:48.150464+0000","last_active":"2024-04-26T22:00:44.644453+0000","last_peered":"2024-04-26T22:00:44.644453+0000","last_clean":"2024-04-26T22:00:44.644453+0000","last_became_active":"2024-04-26T21:59:48.150095+0000","last_became_peered":"2024-04-26T21:59:48.150095+0000","last_unstale":"2024-04-26T22:00:44.644453+0000","last_undegraded":"2024-04-26T22:00:44.644453+0000","last_fullsized":"2024-04-26T22:00:44.644453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:53.152840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":234,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:01:34.756692+0000","last_change":"2024-04-26T21:59:43.038751+0000","last_active":"2024-04-26T22:01:34.756692+0000","last_peered":"2024-04-26T22:01:34.756692+0000","last_clean":"2024-04-26T22:01:34.756692+0000","last_became_active":"2024-04-26T21:59:43.038337+0000","last_became_peered":"2024-04-26T21:59:43.038337+0000","last_unstale":"2024-04-26T22:01:34.756692+0000","last_undegraded":"2024-04-26T22:01:34.756692+0000","last_fullsized":"2024-04-26T22:01:34.756692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.830357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55676,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":179,"num_read_kb":144,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639924+0000","last_change":"2024-04-26T21:59:44.040170+0000","last_active":"2024-04-26T22:00:44.639924+0000","last_peered":"2024-04-26T22:00:44.639924+0000","last_clean":"2024-04-26T22:00:44.639924+0000","last_became_active":"2024-04-26T21:59:44.039713+0000","last_became_peered":"2024-04-26T21:59:44.039713+0000","last_unstale":"2024-04-26T22:00:44.639924+0000","last_undegraded":"2024-04-26T22:00:44.639924+0000","last_fullsized":"2024-04-26T22:00:44.639924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:49.036612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.665987+0000","last_change":"2024-04-26T22:00:44.665631+0000","last_active":"2024-04-26T22:00:44.665987+0000","last_peered":"2024-04-26T22:00:44.665987+0000","last_clean":"2024-04-26T22:00:44.665987+0000","last_became_active":"2024-04-26T22:00:44.665350+0000","last_became_peered":"2024-04-26T22:00:44.665350+0000","last_unstale":"2024-04-26T22:00:44.665987+0000","last_undegraded":"2024-04-26T22:00:44.665987+0000","last_fullsized":"2024-04-26T22:00:44.665987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:54.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900501+0000","last_change":"2024-04-26T21:59:43.039994+0000","last_active":"2024-04-26T22:00:44.900501+0000","last_peered":"2024-04-26T22:00:44.900501+0000","last_clean":"2024-04-26T22:00:44.900501+0000","last_became_active":"2024-04-26T21:59:43.039671+0000","last_became_peered":"2024-04-26T21:59:43.039671+0000","last_unstale":"2024-04-26T22:00:44.900501+0000","last_undegraded":"2024-04-26T22:00:44.900501+0000","last_fullsized":"2024-04-26T22:00:44.900501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:59.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646609+0000","last_change":"2024-04-26T21:59:44.065515+0000","last_active":"2024-04-26T22:00:44.646609+0000","last_peered":"2024-04-26T22:00:44.646609+0000","last_clean":"2024-04-26T22:00:44.646609+0000","last_became_active":"2024-04-26T21:59:44.065120+0000","last_became_peered":"2024-04-26T21:59:44.065120+0000","last_unstale":"2024-04-26T22:00:44.646609+0000","last_undegraded":"2024-04-26T22:00:44.646609+0000","last_fullsized":"2024-04-26T22:00:44.646609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:09.709198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647004+0000","last_change":"2024-04-26T21:59:48.149372+0000","last_active":"2024-04-26T22:00:44.647004+0000","last_peered":"2024-04-26T22:00:44.647004+0000","last_clean":"2024-04-26T22:00:44.647004+0000","last_became_active":"2024-04-26T21:59:48.148920+0000","last_became_peered":"2024-04-26T21:59:48.148920+0000","last_unstale":"2024-04-26T22:00:44.647004+0000","last_undegraded":"2024-04-26T22:00:44.647004+0000","last_fullsized":"2024-04-26T22:00:44.647004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:46.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647893+0000","last_change":"2024-04-26T21:59:43.040576+0000","last_active":"2024-04-26T22:00:44.647893+0000","last_peered":"2024-04-26T22:00:44.647893+0000","last_clean":"2024-04-26T22:00:44.647893+0000","last_became_active":"2024-04-26T21:59:43.040217+0000","last_became_peered":"2024-04-26T21:59:43.040217+0000","last_unstale":"2024-04-26T22:00:44.647893+0000","last_undegraded":"2024-04-26T22:00:44.647893+0000","last_fullsized":"2024-04-26T22:00:44.647893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:58.860607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644712+0000","last_change":"2024-04-26T21:59:44.067746+0000","last_active":"2024-04-26T22:00:44.644712+0000","last_peered":"2024-04-26T22:00:44.644712+0000","last_clean":"2024-04-26T22:00:44.644712+0000","last_became_active":"2024-04-26T21:59:44.067434+0000","last_became_peered":"2024-04-26T21:59:44.067434+0000","last_unstale":"2024-04-26T22:00:44.644712+0000","last_undegraded":"2024-04-26T22:00:44.644712+0000","last_fullsized":"2024-04-26T22:00:44.644712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:45.304899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647057+0000","last_change":"2024-04-26T21:59:48.157380+0000","last_active":"2024-04-26T22:00:44.647057+0000","last_peered":"2024-04-26T22:00:44.647057+0000","last_clean":"2024-04-26T22:00:44.647057+0000","last_became_active":"2024-04-26T21:59:48.155887+0000","last_became_peered":"2024-04-26T21:59:48.155887+0000","last_unstale":"2024-04-26T22:00:44.647057+0000","last_undegraded":"2024-04-26T22:00:44.647057+0000","last_fullsized":"2024-04-26T22:00:44.647057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.339191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640385+0000","last_change":"2024-04-26T21:59:43.036773+0000","last_active":"2024-04-26T22:00:44.640385+0000","last_peered":"2024-04-26T22:00:44.640385+0000","last_clean":"2024-04-26T22:00:44.640385+0000","last_became_active":"2024-04-26T21:59:43.036384+0000","last_became_peered":"2024-04-26T21:59:43.036384+0000","last_unstale":"2024-04-26T22:00:44.640385+0000","last_undegraded":"2024-04-26T22:00:44.640385+0000","last_fullsized":"2024-04-26T22:00:44.640385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:33.841309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.661062+0000","last_change":"2024-04-26T22:00:44.661062+0000","last_active":"2024-04-26T22:00:44.661062+0000","last_peered":"2024-04-26T22:00:44.661062+0000","last_clean":"2024-04-26T22:00:44.661062+0000","last_became_active":"2024-04-26T22:00:44.660471+0000","last_became_peered":"2024-04-26T22:00:44.660471+0000","last_unstale":"2024-04-26T22:00:44.661062+0000","last_undegraded":"2024-04-26T22:00:44.661062+0000","last_fullsized":"2024-04-26T22:00:44.661062+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:30.077158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644190+0000","last_change":"2024-04-26T21:59:48.156690+0000","last_active":"2024-04-26T22:00:44.644190+0000","last_peered":"2024-04-26T22:00:44.644190+0000","last_clean":"2024-04-26T22:00:44.644190+0000","last_became_active":"2024-04-26T21:59:48.155067+0000","last_became_peered":"2024-04-26T21:59:48.155067+0000","last_unstale":"2024-04-26T22:00:44.644190+0000","last_undegraded":"2024-04-26T22:00:44.644190+0000","last_fullsized":"2024-04-26T22:00:44.644190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:39.366540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639430+0000","last_change":"2024-04-26T21:59:43.050964+0000","last_active":"2024-04-26T22:00:44.639430+0000","last_peered":"2024-04-26T22:00:44.639430+0000","last_clean":"2024-04-26T22:00:44.639430+0000","last_became_active":"2024-04-26T21:59:43.050622+0000","last_became_peered":"2024-04-26T21:59:43.050622+0000","last_unstale":"2024-04-26T22:00:44.639430+0000","last_undegraded":"2024-04-26T22:00:44.639430+0000","last_fullsized":"2024-04-26T22:00:44.639430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:18.246708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646546+0000","last_change":"2024-04-26T21:59:44.044126+0000","last_active":"2024-04-26T22:00:44.646546+0000","last_peered":"2024-04-26T22:00:44.646546+0000","last_clean":"2024-04-26T22:00:44.646546+0000","last_became_active":"2024-04-26T21:59:44.043800+0000","last_became_peered":"2024-04-26T21:59:44.043800+0000","last_unstale":"2024-04-26T22:00:44.646546+0000","last_undegraded":"2024-04-26T22:00:44.646546+0000","last_fullsized":"2024-04-26T22:00:44.646546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:46.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646388+0000","last_change":"2024-04-26T21:59:48.134868+0000","last_active":"2024-04-26T22:00:44.646388+0000","last_peered":"2024-04-26T22:00:44.646388+0000","last_clean":"2024-04-26T22:00:44.646388+0000","last_became_active":"2024-04-26T21:59:48.134545+0000","last_became_peered":"2024-04-26T21:59:48.134545+0000","last_unstale":"2024-04-26T22:00:44.646388+0000","last_undegraded":"2024-04-26T22:00:44.646388+0000","last_fullsized":"2024-04-26T22:00:44.646388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:59.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640177+0000","last_change":"2024-04-26T21:59:43.038912+0000","last_active":"2024-04-26T22:00:44.640177+0000","last_peered":"2024-04-26T22:00:44.640177+0000","last_clean":"2024-04-26T22:00:44.640177+0000","last_became_active":"2024-04-26T21:59:43.038597+0000","last_became_peered":"2024-04-26T21:59:43.038597+0000","last_unstale":"2024-04-26T22:00:44.640177+0000","last_undegraded":"2024-04-26T22:00:44.640177+0000","last_fullsized":"2024-04-26T22:00:44.640177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:37.087595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648113+0000","last_change":"2024-04-26T21:59:44.043297+0000","last_active":"2024-04-26T22:00:44.648113+0000","last_peered":"2024-04-26T22:00:44.648113+0000","last_clean":"2024-04-26T22:00:44.648113+0000","last_became_active":"2024-04-26T21:59:44.042952+0000","last_became_peered":"2024-04-26T21:59:44.042952+0000","last_unstale":"2024-04-26T22:00:44.648113+0000","last_undegraded":"2024-04-26T22:00:44.648113+0000","last_fullsized":"2024-04-26T22:00:44.648113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:18.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639597+0000","last_change":"2024-04-26T21:59:48.155119+0000","last_active":"2024-04-26T22:00:44.639597+0000","last_peered":"2024-04-26T22:00:44.639597+0000","last_clean":"2024-04-26T22:00:44.639597+0000","last_became_active":"2024-04-26T21:59:48.154593+0000","last_became_peered":"2024-04-26T21:59:48.154593+0000","last_unstale":"2024-04-26T22:00:44.639597+0000","last_undegraded":"2024-04-26T22:00:44.639597+0000","last_fullsized":"2024-04-26T22:00:44.639597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:01.896610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646386+0000","last_change":"2024-04-26T21:59:43.067817+0000","last_active":"2024-04-26T22:00:44.646386+0000","last_peered":"2024-04-26T22:00:44.646386+0000","last_clean":"2024-04-26T22:00:44.646386+0000","last_became_active":"2024-04-26T21:59:43.067450+0000","last_became_peered":"2024-04-26T21:59:43.067450+0000","last_unstale":"2024-04-26T22:00:44.646386+0000","last_undegraded":"2024-04-26T22:00:44.646386+0000","last_fullsized":"2024-04-26T22:00:44.646386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:29.425903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643412+0000","last_change":"2024-04-26T21:59:44.055085+0000","last_active":"2024-04-26T22:00:44.643412+0000","last_peered":"2024-04-26T22:00:44.643412+0000","last_clean":"2024-04-26T22:00:44.643412+0000","last_became_active":"2024-04-26T21:59:44.054627+0000","last_became_peered":"2024-04-26T21:59:44.054627+0000","last_unstale":"2024-04-26T22:00:44.643412+0000","last_undegraded":"2024-04-26T22:00:44.643412+0000","last_fullsized":"2024-04-26T22:00:44.643412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.234287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645845+0000","last_change":"2024-04-26T21:59:48.147333+0000","last_active":"2024-04-26T22:00:44.645845+0000","last_peered":"2024-04-26T22:00:44.645845+0000","last_clean":"2024-04-26T22:00:44.645845+0000","last_became_active":"2024-04-26T21:59:48.146926+0000","last_became_peered":"2024-04-26T21:59:48.146926+0000","last_unstale":"2024-04-26T22:00:44.645845+0000","last_undegraded":"2024-04-26T22:00:44.645845+0000","last_fullsized":"2024-04-26T22:00:44.645845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644463+0000","last_change":"2024-04-26T21:59:43.050434+0000","last_active":"2024-04-26T22:00:44.644463+0000","last_peered":"2024-04-26T22:00:44.644463+0000","last_clean":"2024-04-26T22:00:44.644463+0000","last_became_active":"2024-04-26T21:59:43.050096+0000","last_became_peered":"2024-04-26T21:59:43.050096+0000","last_unstale":"2024-04-26T22:00:44.644463+0000","last_undegraded":"2024-04-26T22:00:44.644463+0000","last_fullsized":"2024-04-26T22:00:44.644463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:22.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643170+0000","last_change":"2024-04-26T21:59:44.048302+0000","last_active":"2024-04-26T22:00:44.643170+0000","last_peered":"2024-04-26T22:00:44.643170+0000","last_clean":"2024-04-26T22:00:44.643170+0000","last_became_active":"2024-04-26T21:59:44.047898+0000","last_became_peered":"2024-04-26T21:59:44.047898+0000","last_unstale":"2024-04-26T22:00:44.643170+0000","last_undegraded":"2024-04-26T22:00:44.643170+0000","last_fullsized":"2024-04-26T22:00:44.643170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:55.718558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646834+0000","last_change":"2024-04-26T21:59:48.148025+0000","last_active":"2024-04-26T22:00:44.646834+0000","last_peered":"2024-04-26T22:00:44.646834+0000","last_clean":"2024-04-26T22:00:44.646834+0000","last_became_active":"2024-04-26T21:59:48.147065+0000","last_became_peered":"2024-04-26T21:59:48.147065+0000","last_unstale":"2024-04-26T22:00:44.646834+0000","last_undegraded":"2024-04-26T22:00:44.646834+0000","last_fullsized":"2024-04-26T22:00:44.646834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:23.932501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646753+0000","last_change":"2024-04-26T21:59:44.045186+0000","last_active":"2024-04-26T22:00:44.646753+0000","last_peered":"2024-04-26T22:00:44.646753+0000","last_clean":"2024-04-26T22:00:44.646753+0000","last_became_active":"2024-04-26T21:59:44.044761+0000","last_became_peered":"2024-04-26T21:59:44.044761+0000","last_unstale":"2024-04-26T22:00:44.646753+0000","last_undegraded":"2024-04-26T22:00:44.646753+0000","last_fullsized":"2024-04-26T22:00:44.646753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:32.744412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645421+0000","last_change":"2024-04-26T21:59:43.068695+0000","last_active":"2024-04-26T22:00:44.645421+0000","last_peered":"2024-04-26T22:00:44.645421+0000","last_clean":"2024-04-26T22:00:44.645421+0000","last_became_active":"2024-04-26T21:59:43.068210+0000","last_became_peered":"2024-04-26T21:59:43.068210+0000","last_unstale":"2024-04-26T22:00:44.645421+0000","last_undegraded":"2024-04-26T22:00:44.645421+0000","last_fullsized":"2024-04-26T22:00:44.645421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:19.914872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.663734+0000","last_change":"2024-04-26T22:00:44.663734+0000","last_active":"2024-04-26T22:00:44.663734+0000","last_peered":"2024-04-26T22:00:44.663734+0000","last_clean":"2024-04-26T22:00:44.663734+0000","last_became_active":"2024-04-26T22:00:44.649296+0000","last_became_peered":"2024-04-26T22:00:44.649296+0000","last_unstale":"2024-04-26T22:00:44.663734+0000","last_undegraded":"2024-04-26T22:00:44.663734+0000","last_fullsized":"2024-04-26T22:00:44.663734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:30.645448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651556+0000","last_change":"2024-04-26T21:59:44.052732+0000","last_active":"2024-04-26T22:00:44.651556+0000","last_peered":"2024-04-26T22:00:44.651556+0000","last_clean":"2024-04-26T22:00:44.651556+0000","last_became_active":"2024-04-26T21:59:44.051202+0000","last_became_peered":"2024-04-26T21:59:44.051202+0000","last_unstale":"2024-04-26T22:00:44.651556+0000","last_undegraded":"2024-04-26T22:00:44.651556+0000","last_fullsized":"2024-04-26T22:00:44.651556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:20.887239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641223+0000","last_change":"2024-04-26T21:59:43.054776+0000","last_active":"2024-04-26T22:00:44.641223+0000","last_peered":"2024-04-26T22:00:44.641223+0000","last_clean":"2024-04-26T22:00:44.641223+0000","last_became_active":"2024-04-26T21:59:43.054410+0000","last_became_peered":"2024-04-26T21:59:43.054410+0000","last_unstale":"2024-04-26T22:00:44.641223+0000","last_undegraded":"2024-04-26T22:00:44.641223+0000","last_fullsized":"2024-04-26T22:00:44.641223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:11.675134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644238+0000","last_change":"2024-04-26T21:58:22.546842+0000","last_active":"2024-04-26T22:00:44.644238+0000","last_peered":"2024-04-26T22:00:44.644238+0000","last_clean":"2024-04-26T22:00:44.644238+0000","last_became_active":"2024-04-26T21:58:22.546448+0000","last_became_peered":"2024-04-26T21:58:22.546448+0000","last_unstale":"2024-04-26T22:00:44.644238+0000","last_undegraded":"2024-04-26T22:00:44.644238+0000","last_fullsized":"2024-04-26T22:00:44.644238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_clean_scrub_stamp":"2024-04-26T21:54:29.502062+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:31.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646875+0000","last_change":"2024-04-26T21:59:48.152986+0000","last_active":"2024-04-26T22:00:44.646875+0000","last_peered":"2024-04-26T22:00:44.646875+0000","last_clean":"2024-04-26T22:00:44.646875+0000","last_became_active":"2024-04-26T21:59:48.152606+0000","last_became_peered":"2024-04-26T21:59:48.152606+0000","last_unstale":"2024-04-26T22:00:44.646875+0000","last_undegraded":"2024-04-26T22:00:44.646875+0000","last_fullsized":"2024-04-26T22:00:44.646875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645256+0000","last_change":"2024-04-26T21:59:44.060086+0000","last_active":"2024-04-26T22:00:44.645256+0000","last_peered":"2024-04-26T22:00:44.645256+0000","last_clean":"2024-04-26T22:00:44.645256+0000","last_became_active":"2024-04-26T21:59:44.059742+0000","last_became_peered":"2024-04-26T21:59:44.059742+0000","last_unstale":"2024-04-26T22:00:44.645256+0000","last_undegraded":"2024-04-26T22:00:44.645256+0000","last_fullsized":"2024-04-26T22:00:44.645256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.678769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644321+0000","last_change":"2024-04-26T21:59:43.034726+0000","last_active":"2024-04-26T22:00:44.644321+0000","last_peered":"2024-04-26T22:00:44.644321+0000","last_clean":"2024-04-26T22:00:44.644321+0000","last_became_active":"2024-04-26T21:59:43.034407+0000","last_became_peered":"2024-04-26T21:59:43.034407+0000","last_unstale":"2024-04-26T22:00:44.644321+0000","last_undegraded":"2024-04-26T22:00:44.644321+0000","last_fullsized":"2024-04-26T22:00:44.644321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:37.722860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647601+0000","last_change":"2024-04-26T21:59:48.136044+0000","last_active":"2024-04-26T22:00:44.647601+0000","last_peered":"2024-04-26T22:00:44.647601+0000","last_clean":"2024-04-26T22:00:44.647601+0000","last_became_active":"2024-04-26T21:59:48.135619+0000","last_became_peered":"2024-04-26T21:59:48.135619+0000","last_unstale":"2024-04-26T22:00:44.647601+0000","last_undegraded":"2024-04-26T22:00:44.647601+0000","last_fullsized":"2024-04-26T22:00:44.647601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:24.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639995+0000","last_change":"2024-04-26T21:59:43.059149+0000","last_active":"2024-04-26T22:00:44.639995+0000","last_peered":"2024-04-26T22:00:44.639995+0000","last_clean":"2024-04-26T22:00:44.639995+0000","last_became_active":"2024-04-26T21:59:43.058816+0000","last_became_peered":"2024-04-26T21:59:43.058816+0000","last_unstale":"2024-04-26T22:00:44.639995+0000","last_undegraded":"2024-04-26T22:00:44.639995+0000","last_fullsized":"2024-04-26T22:00:44.639995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:04.441843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646064+0000","last_change":"2024-04-26T21:59:44.063544+0000","last_active":"2024-04-26T22:00:44.646064+0000","last_peered":"2024-04-26T22:00:44.646064+0000","last_clean":"2024-04-26T22:00:44.646064+0000","last_became_active":"2024-04-26T21:59:44.063179+0000","last_became_peered":"2024-04-26T21:59:44.063179+0000","last_unstale":"2024-04-26T22:00:44.646064+0000","last_undegraded":"2024-04-26T22:00:44.646064+0000","last_fullsized":"2024-04-26T22:00:44.646064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:44.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650531+0000","last_change":"2024-04-26T21:59:48.131695+0000","last_active":"2024-04-26T22:00:44.650531+0000","last_peered":"2024-04-26T22:00:44.650531+0000","last_clean":"2024-04-26T22:00:44.650531+0000","last_became_active":"2024-04-26T21:59:48.131246+0000","last_became_peered":"2024-04-26T21:59:48.131246+0000","last_unstale":"2024-04-26T22:00:44.650531+0000","last_undegraded":"2024-04-26T22:00:44.650531+0000","last_fullsized":"2024-04-26T22:00:44.650531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:53.128553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647819+0000","last_change":"2024-04-26T21:59:43.067463+0000","last_active":"2024-04-26T22:00:44.647819+0000","last_peered":"2024-04-26T22:00:44.647819+0000","last_clean":"2024-04-26T22:00:44.647819+0000","last_became_active":"2024-04-26T21:59:43.067009+0000","last_became_peered":"2024-04-26T21:59:43.067009+0000","last_unstale":"2024-04-26T22:00:44.647819+0000","last_undegraded":"2024-04-26T22:00:44.647819+0000","last_fullsized":"2024-04-26T22:00:44.647819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:35.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.658109+0000","last_change":"2024-04-26T21:59:44.054781+0000","last_active":"2024-04-26T22:00:44.658109+0000","last_peered":"2024-04-26T22:00:44.658109+0000","last_clean":"2024-04-26T22:00:44.658109+0000","last_became_active":"2024-04-26T21:59:44.049439+0000","last_became_peered":"2024-04-26T21:59:44.049439+0000","last_unstale":"2024-04-26T22:00:44.658109+0000","last_undegraded":"2024-04-26T22:00:44.658109+0000","last_fullsized":"2024-04-26T22:00:44.658109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:50.523835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900373+0000","last_change":"2024-04-26T21:59:48.127776+0000","last_active":"2024-04-26T22:00:44.900373+0000","last_peered":"2024-04-26T22:00:44.900373+0000","last_clean":"2024-04-26T22:00:44.900373+0000","last_became_active":"2024-04-26T21:59:48.127207+0000","last_became_peered":"2024-04-26T21:59:48.127207+0000","last_unstale":"2024-04-26T22:00:44.900373+0000","last_undegraded":"2024-04-26T22:00:44.900373+0000","last_fullsized":"2024-04-26T22:00:44.900373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:23.794371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641087+0000","last_change":"2024-04-26T21:59:43.050320+0000","last_active":"2024-04-26T22:00:44.641087+0000","last_peered":"2024-04-26T22:00:44.641087+0000","last_clean":"2024-04-26T22:00:44.641087+0000","last_became_active":"2024-04-26T21:59:43.049890+0000","last_became_peered":"2024-04-26T21:59:43.049890+0000","last_unstale":"2024-04-26T22:00:44.641087+0000","last_undegraded":"2024-04-26T22:00:44.641087+0000","last_fullsized":"2024-04-26T22:00:44.641087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:53.589776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657485+0000","last_change":"2024-04-26T21:59:44.057851+0000","last_active":"2024-04-26T22:00:44.657485+0000","last_peered":"2024-04-26T22:00:44.657485+0000","last_clean":"2024-04-26T22:00:44.657485+0000","last_became_active":"2024-04-26T21:59:44.057525+0000","last_became_peered":"2024-04-26T21:59:44.057525+0000","last_unstale":"2024-04-26T22:00:44.657485+0000","last_undegraded":"2024-04-26T22:00:44.657485+0000","last_fullsized":"2024-04-26T22:00:44.657485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:30.986251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640401+0000","last_change":"2024-04-26T21:59:48.154962+0000","last_active":"2024-04-26T22:00:44.640401+0000","last_peered":"2024-04-26T22:00:44.640401+0000","last_clean":"2024-04-26T22:00:44.640401+0000","last_became_active":"2024-04-26T21:59:48.154479+0000","last_became_peered":"2024-04-26T21:59:48.154479+0000","last_unstale":"2024-04-26T22:00:44.640401+0000","last_undegraded":"2024-04-26T22:00:44.640401+0000","last_fullsized":"2024-04-26T22:00:44.640401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:30.415432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900323+0000","last_change":"2024-04-26T21:59:43.051186+0000","last_active":"2024-04-26T22:00:44.900323+0000","last_peered":"2024-04-26T22:00:44.900323+0000","last_clean":"2024-04-26T22:00:44.900323+0000","last_became_active":"2024-04-26T21:59:43.050768+0000","last_became_peered":"2024-04-26T21:59:43.050768+0000","last_unstale":"2024-04-26T22:00:44.900323+0000","last_undegraded":"2024-04-26T22:00:44.900323+0000","last_fullsized":"2024-04-26T22:00:44.900323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.237229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639400+0000","last_change":"2024-04-26T21:59:44.039113+0000","last_active":"2024-04-26T22:00:44.639400+0000","last_peered":"2024-04-26T22:00:44.639400+0000","last_clean":"2024-04-26T22:00:44.639400+0000","last_became_active":"2024-04-26T21:59:44.038610+0000","last_became_peered":"2024-04-26T21:59:44.038610+0000","last_unstale":"2024-04-26T22:00:44.639400+0000","last_undegraded":"2024-04-26T22:00:44.639400+0000","last_fullsized":"2024-04-26T22:00:44.639400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:39.450603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647252+0000","last_change":"2024-04-26T21:59:48.157058+0000","last_active":"2024-04-26T22:00:44.647252+0000","last_peered":"2024-04-26T22:00:44.647252+0000","last_clean":"2024-04-26T22:00:44.647252+0000","last_became_active":"2024-04-26T21:59:48.153897+0000","last_became_peered":"2024-04-26T21:59:48.153897+0000","last_unstale":"2024-04-26T22:00:44.647252+0000","last_undegraded":"2024-04-26T22:00:44.647252+0000","last_fullsized":"2024-04-26T22:00:44.647252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.044493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651113+0000","last_change":"2024-04-26T21:59:43.061101+0000","last_active":"2024-04-26T22:00:44.651113+0000","last_peered":"2024-04-26T22:00:44.651113+0000","last_clean":"2024-04-26T22:00:44.651113+0000","last_became_active":"2024-04-26T21:59:43.060762+0000","last_became_peered":"2024-04-26T21:59:43.060762+0000","last_unstale":"2024-04-26T22:00:44.651113+0000","last_undegraded":"2024-04-26T22:00:44.651113+0000","last_fullsized":"2024-04-26T22:00:44.651113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:07.961535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647073+0000","last_change":"2024-04-26T21:59:44.047437+0000","last_active":"2024-04-26T22:00:44.647073+0000","last_peered":"2024-04-26T22:00:44.647073+0000","last_clean":"2024-04-26T22:00:44.647073+0000","last_became_active":"2024-04-26T21:59:44.047127+0000","last_became_peered":"2024-04-26T21:59:44.047127+0000","last_unstale":"2024-04-26T22:00:44.647073+0000","last_undegraded":"2024-04-26T22:00:44.647073+0000","last_fullsized":"2024-04-26T22:00:44.647073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:12.259965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.660193+0000","last_change":"2024-04-26T22:00:44.660193+0000","last_active":"2024-04-26T22:00:44.660193+0000","last_peered":"2024-04-26T22:00:44.660193+0000","last_clean":"2024-04-26T22:00:44.660193+0000","last_became_active":"2024-04-26T22:00:44.645540+0000","last_became_peered":"2024-04-26T22:00:44.645540+0000","last_unstale":"2024-04-26T22:00:44.660193+0000","last_undegraded":"2024-04-26T22:00:44.660193+0000","last_fullsized":"2024-04-26T22:00:44.660193+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:03.437665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900467+0000","last_change":"2024-04-26T21:59:43.043901+0000","last_active":"2024-04-26T22:00:44.900467+0000","last_peered":"2024-04-26T22:00:44.900467+0000","last_clean":"2024-04-26T22:00:44.900467+0000","last_became_active":"2024-04-26T21:59:43.043572+0000","last_became_peered":"2024-04-26T21:59:43.043572+0000","last_unstale":"2024-04-26T22:00:44.900467+0000","last_undegraded":"2024-04-26T22:00:44.900467+0000","last_fullsized":"2024-04-26T22:00:44.900467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.107295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656522+0000","last_change":"2024-04-26T21:59:44.059451+0000","last_active":"2024-04-26T22:00:44.656522+0000","last_peered":"2024-04-26T22:00:44.656522+0000","last_clean":"2024-04-26T22:00:44.656522+0000","last_became_active":"2024-04-26T21:59:44.056771+0000","last_became_peered":"2024-04-26T21:59:44.056771+0000","last_unstale":"2024-04-26T22:00:44.656522+0000","last_undegraded":"2024-04-26T22:00:44.656522+0000","last_fullsized":"2024-04-26T22:00:44.656522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:53.641182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640389+0000","last_change":"2024-04-26T21:59:48.130222+0000","last_active":"2024-04-26T22:00:44.640389+0000","last_peered":"2024-04-26T22:00:44.640389+0000","last_clean":"2024-04-26T22:00:44.640389+0000","last_became_active":"2024-04-26T21:59:48.129379+0000","last_became_peered":"2024-04-26T21:59:48.129379+0000","last_unstale":"2024-04-26T22:00:44.640389+0000","last_undegraded":"2024-04-26T22:00:44.640389+0000","last_fullsized":"2024-04-26T22:00:44.640389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:26.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639569+0000","last_change":"2024-04-26T21:59:43.035503+0000","last_active":"2024-04-26T22:00:44.639569+0000","last_peered":"2024-04-26T22:00:44.639569+0000","last_clean":"2024-04-26T22:00:44.639569+0000","last_became_active":"2024-04-26T21:59:43.035180+0000","last_became_peered":"2024-04-26T21:59:43.035180+0000","last_unstale":"2024-04-26T22:00:44.639569+0000","last_undegraded":"2024-04-26T22:00:44.639569+0000","last_fullsized":"2024-04-26T22:00:44.639569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:45.696707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644574+0000","last_change":"2024-04-26T21:59:44.054444+0000","last_active":"2024-04-26T22:00:44.644574+0000","last_peered":"2024-04-26T22:00:44.644574+0000","last_clean":"2024-04-26T22:00:44.644574+0000","last_became_active":"2024-04-26T21:59:44.054035+0000","last_became_peered":"2024-04-26T21:59:44.054035+0000","last_unstale":"2024-04-26T22:00:44.644574+0000","last_undegraded":"2024-04-26T22:00:44.644574+0000","last_fullsized":"2024-04-26T22:00:44.644574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:04.905286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900354+0000","last_change":"2024-04-26T21:59:48.141551+0000","last_active":"2024-04-26T22:00:44.900354+0000","last_peered":"2024-04-26T22:00:44.900354+0000","last_clean":"2024-04-26T22:00:44.900354+0000","last_became_active":"2024-04-26T21:59:48.141166+0000","last_became_peered":"2024-04-26T21:59:48.141166+0000","last_unstale":"2024-04-26T22:00:44.900354+0000","last_undegraded":"2024-04-26T22:00:44.900354+0000","last_fullsized":"2024-04-26T22:00:44.900354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:38.810387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900537+0000","last_change":"2024-04-26T21:59:43.059420+0000","last_active":"2024-04-26T22:00:44.900537+0000","last_peered":"2024-04-26T22:00:44.900537+0000","last_clean":"2024-04-26T22:00:44.900537+0000","last_became_active":"2024-04-26T21:59:43.059033+0000","last_became_peered":"2024-04-26T21:59:43.059033+0000","last_unstale":"2024-04-26T22:00:44.900537+0000","last_undegraded":"2024-04-26T22:00:44.900537+0000","last_fullsized":"2024-04-26T22:00:44.900537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:57.633543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645231+0000","last_change":"2024-04-26T21:59:44.045932+0000","last_active":"2024-04-26T22:00:44.645231+0000","last_peered":"2024-04-26T22:00:44.645231+0000","last_clean":"2024-04-26T22:00:44.645231+0000","last_became_active":"2024-04-26T21:59:44.044775+0000","last_became_peered":"2024-04-26T21:59:44.044775+0000","last_unstale":"2024-04-26T22:00:44.645231+0000","last_undegraded":"2024-04-26T22:00:44.645231+0000","last_fullsized":"2024-04-26T22:00:44.645231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:05.719002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.655070+0000","last_change":"2024-04-26T21:59:48.151182+0000","last_active":"2024-04-26T22:00:44.655070+0000","last_peered":"2024-04-26T22:00:44.655070+0000","last_clean":"2024-04-26T22:00:44.655070+0000","last_became_active":"2024-04-26T21:59:48.150815+0000","last_became_peered":"2024-04-26T21:59:48.150815+0000","last_unstale":"2024-04-26T22:00:44.655070+0000","last_undegraded":"2024-04-26T22:00:44.655070+0000","last_fullsized":"2024-04-26T22:00:44.655070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.630163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647186+0000","last_change":"2024-04-26T21:59:43.039852+0000","last_active":"2024-04-26T22:00:44.647186+0000","last_peered":"2024-04-26T22:00:44.647186+0000","last_clean":"2024-04-26T22:00:44.647186+0000","last_became_active":"2024-04-26T21:59:43.039499+0000","last_became_peered":"2024-04-26T21:59:43.039499+0000","last_unstale":"2024-04-26T22:00:44.647186+0000","last_undegraded":"2024-04-26T22:00:44.647186+0000","last_fullsized":"2024-04-26T22:00:44.647186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:14.599270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644898+0000","last_change":"2024-04-26T21:59:44.057285+0000","last_active":"2024-04-26T22:00:44.644898+0000","last_peered":"2024-04-26T22:00:44.644898+0000","last_clean":"2024-04-26T22:00:44.644898+0000","last_became_active":"2024-04-26T21:59:44.056947+0000","last_became_peered":"2024-04-26T21:59:44.056947+0000","last_unstale":"2024-04-26T22:00:44.644898+0000","last_undegraded":"2024-04-26T22:00:44.644898+0000","last_fullsized":"2024-04-26T22:00:44.644898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:58.242075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646714+0000","last_change":"2024-04-26T21:59:48.131624+0000","last_active":"2024-04-26T22:00:44.646714+0000","last_peered":"2024-04-26T22:00:44.646714+0000","last_clean":"2024-04-26T22:00:44.646714+0000","last_became_active":"2024-04-26T21:59:48.131193+0000","last_became_peered":"2024-04-26T21:59:48.131193+0000","last_unstale":"2024-04-26T22:00:44.646714+0000","last_undegraded":"2024-04-26T22:00:44.646714+0000","last_fullsized":"2024-04-26T22:00:44.646714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:52.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643880+0000","last_change":"2024-04-26T21:59:43.047723+0000","last_active":"2024-04-26T22:00:44.643880+0000","last_peered":"2024-04-26T22:00:44.643880+0000","last_clean":"2024-04-26T22:00:44.643880+0000","last_became_active":"2024-04-26T21:59:43.045876+0000","last_became_peered":"2024-04-26T21:59:43.045876+0000","last_unstale":"2024-04-26T22:00:44.643880+0000","last_undegraded":"2024-04-26T22:00:44.643880+0000","last_fullsized":"2024-04-26T22:00:44.643880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:37.705490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645171+0000","last_change":"2024-04-26T21:59:44.040723+0000","last_active":"2024-04-26T22:00:44.645171+0000","last_peered":"2024-04-26T22:00:44.645171+0000","last_clean":"2024-04-26T22:00:44.645171+0000","last_became_active":"2024-04-26T21:59:44.040389+0000","last_became_peered":"2024-04-26T21:59:44.040389+0000","last_unstale":"2024-04-26T22:00:44.645171+0000","last_undegraded":"2024-04-26T22:00:44.645171+0000","last_fullsized":"2024-04-26T22:00:44.645171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:05.120695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647186+0000","last_change":"2024-04-26T21:59:44.041027+0000","last_active":"2024-04-26T22:00:44.647186+0000","last_peered":"2024-04-26T22:00:44.647186+0000","last_clean":"2024-04-26T22:00:44.647186+0000","last_became_active":"2024-04-26T21:59:44.040705+0000","last_became_peered":"2024-04-26T21:59:44.040705+0000","last_unstale":"2024-04-26T22:00:44.647186+0000","last_undegraded":"2024-04-26T22:00:44.647186+0000","last_fullsized":"2024-04-26T22:00:44.647186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:11.114850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900928+0000","last_change":"2024-04-26T21:59:44.040916+0000","last_active":"2024-04-26T22:00:44.900928+0000","last_peered":"2024-04-26T22:00:44.900928+0000","last_clean":"2024-04-26T22:00:44.900928+0000","last_became_active":"2024-04-26T21:59:44.040568+0000","last_became_peered":"2024-04-26T21:59:44.040568+0000","last_unstale":"2024-04-26T22:00:44.900928+0000","last_undegraded":"2024-04-26T22:00:44.900928+0000","last_fullsized":"2024-04-26T22:00:44.900928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:04.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900357+0000","last_change":"2024-04-26T21:59:44.046676+0000","last_active":"2024-04-26T22:00:44.900357+0000","last_peered":"2024-04-26T22:00:44.900357+0000","last_clean":"2024-04-26T22:00:44.900357+0000","last_became_active":"2024-04-26T21:59:44.046339+0000","last_became_peered":"2024-04-26T21:59:44.046339+0000","last_unstale":"2024-04-26T22:00:44.900357+0000","last_undegraded":"2024-04-26T22:00:44.900357+0000","last_fullsized":"2024-04-26T22:00:44.900357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:51.964558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.655771+0000","last_change":"2024-04-26T21:59:44.064635+0000","last_active":"2024-04-26T22:00:44.655771+0000","last_peered":"2024-04-26T22:00:44.655771+0000","last_clean":"2024-04-26T22:00:44.655771+0000","last_became_active":"2024-04-26T21:59:44.064174+0000","last_became_peered":"2024-04-26T21:59:44.064174+0000","last_unstale":"2024-04-26T22:00:44.655771+0000","last_undegraded":"2024-04-26T22:00:44.655771+0000","last_fullsized":"2024-04-26T22:00:44.655771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:59.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.666626+0000","last_change":"2024-04-26T22:00:44.666626+0000","last_active":"2024-04-26T22:00:44.666626+0000","last_peered":"2024-04-26T22:00:44.666626+0000","last_clean":"2024-04-26T22:00:44.666626+0000","last_became_active":"2024-04-26T22:00:44.666239+0000","last_became_peered":"2024-04-26T22:00:44.666239+0000","last_unstale":"2024-04-26T22:00:44.666626+0000","last_undegraded":"2024-04-26T22:00:44.666626+0000","last_fullsized":"2024-04-26T22:00:44.666626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:13.684014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648356+0000","last_change":"2024-04-26T21:59:44.044024+0000","last_active":"2024-04-26T22:00:44.648356+0000","last_peered":"2024-04-26T22:00:44.648356+0000","last_clean":"2024-04-26T22:00:44.648356+0000","last_became_active":"2024-04-26T21:59:44.043718+0000","last_became_peered":"2024-04-26T21:59:44.043718+0000","last_unstale":"2024-04-26T22:00:44.648356+0000","last_undegraded":"2024-04-26T22:00:44.648356+0000","last_fullsized":"2024-04-26T22:00:44.648356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:35.028538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.901003+0000","last_change":"2024-04-26T21:59:44.037432+0000","last_active":"2024-04-26T22:00:44.901003+0000","last_peered":"2024-04-26T22:00:44.901003+0000","last_clean":"2024-04-26T22:00:44.901003+0000","last_became_active":"2024-04-26T21:59:44.037086+0000","last_became_peered":"2024-04-26T21:59:44.037086+0000","last_unstale":"2024-04-26T22:00:44.901003+0000","last_undegraded":"2024-04-26T22:00:44.901003+0000","last_fullsized":"2024-04-26T22:00:44.901003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:32.828866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.658899+0000","last_change":"2024-04-26T21:59:44.055253+0000","last_active":"2024-04-26T22:00:44.658899+0000","last_peered":"2024-04-26T22:00:44.658899+0000","last_clean":"2024-04-26T22:00:44.658899+0000","last_became_active":"2024-04-26T21:59:44.054329+0000","last_became_peered":"2024-04-26T21:59:44.054329+0000","last_unstale":"2024-04-26T22:00:44.658899+0000","last_undegraded":"2024-04-26T22:00:44.658899+0000","last_fullsized":"2024-04-26T22:00:44.658899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:02.949272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646654+0000","last_change":"2024-04-26T21:59:44.060213+0000","last_active":"2024-04-26T22:00:44.646654+0000","last_peered":"2024-04-26T22:00:44.646654+0000","last_clean":"2024-04-26T22:00:44.646654+0000","last_became_active":"2024-04-26T21:59:44.059987+0000","last_became_peered":"2024-04-26T21:59:44.059987+0000","last_unstale":"2024-04-26T22:00:44.646654+0000","last_undegraded":"2024-04-26T22:00:44.646654+0000","last_fullsized":"2024-04-26T22:00:44.646654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:23.184373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651222+0000","last_change":"2024-04-26T21:59:44.046779+0000","last_active":"2024-04-26T22:00:44.651222+0000","last_peered":"2024-04-26T22:00:44.651222+0000","last_clean":"2024-04-26T22:00:44.651222+0000","last_became_active":"2024-04-26T21:59:44.046304+0000","last_became_peered":"2024-04-26T21:59:44.046304+0000","last_unstale":"2024-04-26T22:00:44.651222+0000","last_undegraded":"2024-04-26T22:00:44.651222+0000","last_fullsized":"2024-04-26T22:00:44.651222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:59.738908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900530+0000","last_change":"2024-04-26T21:59:44.055284+0000","last_active":"2024-04-26T22:00:44.900530+0000","last_peered":"2024-04-26T22:00:44.900530+0000","last_clean":"2024-04-26T22:00:44.900530+0000","last_became_active":"2024-04-26T21:59:44.054897+0000","last_became_peered":"2024-04-26T21:59:44.054897+0000","last_unstale":"2024-04-26T22:00:44.900530+0000","last_undegraded":"2024-04-26T22:00:44.900530+0000","last_fullsized":"2024-04-26T22:00:44.900530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:57.942834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646627+0000","last_change":"2024-04-26T21:59:44.060516+0000","last_active":"2024-04-26T22:00:44.646627+0000","last_peered":"2024-04-26T22:00:44.646627+0000","last_clean":"2024-04-26T22:00:44.646627+0000","last_became_active":"2024-04-26T21:59:44.059564+0000","last_became_peered":"2024-04-26T21:59:44.059564+0000","last_unstale":"2024-04-26T22:00:44.646627+0000","last_undegraded":"2024-04-26T22:00:44.646627+0000","last_fullsized":"2024-04-26T22:00:44.646627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:43.784018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647813+0000","last_change":"2024-04-26T21:59:44.043564+0000","last_active":"2024-04-26T22:00:44.647813+0000","last_peered":"2024-04-26T22:00:44.647813+0000","last_clean":"2024-04-26T22:00:44.647813+0000","last_became_active":"2024-04-26T21:59:44.043254+0000","last_became_peered":"2024-04-26T21:59:44.043254+0000","last_unstale":"2024-04-26T22:00:44.647813+0000","last_undegraded":"2024-04-26T22:00:44.647813+0000","last_fullsized":"2024-04-26T22:00:44.647813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:12.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56474,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":185,"num_read_kb":150,"num_write":81,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":169422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":504735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1708032,"data_stored":1155069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":501221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1634304,"data_stored":1099781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":556897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":500729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":556349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":500743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1095285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":55710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":55870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:01:37.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T22:01:37.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:36 smithi154 ceph-mon[32578]: pgmap v378: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:36 smithi077 ceph-mon[32280]: pgmap v378: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:38.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:37 smithi012 ceph-mon[28184]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:38.133 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:37 smithi077 ceph-mon[32280]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:38.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:37 smithi154 ceph-mon[32578]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:39.038 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:38 smithi012 ceph-mon[28184]: pgmap v379: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:39.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:38 smithi154 ceph-mon[32578]: pgmap v379: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:39.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:38 smithi077 ceph-mon[32280]: pgmap v379: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 -- 172.21.15.12:0/918486931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 msgr2=0x7fc49c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 --2- 172.21.15.12:0/918486931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1102e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc494009990 tx=0x7fc494009ca0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 -- 172.21.15.12:0/918486931 shutdown_connections 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 --2- 172.21.15.12:0/918486931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1102e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 --2- 172.21.15.12:0/918486931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c1094a0 0x7fc49c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 --2- 172.21.15.12:0/918486931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c06b130 0x7fc49c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 -- 172.21.15.12:0/918486931 >> 172.21.15.12:0/918486931 conn(0x7fc49c0792d0 msgr2=0x7fc49c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 -- 172.21.15.12:0/918486931 shutdown_connections 2024-04-26T22:01:39.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.910+0000 7fc4a2000700 1 -- 172.21.15.12:0/918486931 wait complete. 2024-04-26T22:01:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.911+0000 7fc4a2000700 1 Processor -- start 2024-04-26T22:01:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 -- start start 2024-04-26T22:01:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 0x7fc49c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c1094a0 0x7fc49c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1135c0 con 0x7fc49c06bad0 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c113440 con 0x7fc49c1094a0 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc4a2000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c113740 con 0x7fc49c06b130 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc492a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc492a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56740/0 (socket says 172.21.15.12:56740) 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.912+0000 7fc492a00700 1 -- 172.21.15.12:0/350826432 learned_addr learned my addr 172.21.15.12:0/350826432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc49b400700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 0x7fc49c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc49be00700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c1094a0 0x7fc49c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 -- 172.21.15.12:0/350826432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 msgr2=0x7fc49c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 0x7fc49c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 -- 172.21.15.12:0/350826432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c1094a0 msgr2=0x7fc49c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c1094a0 0x7fc49c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 -- 172.21.15.12:0/350826432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19c450 con 0x7fc49c06bad0 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc49b400700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 0x7fc49c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.913+0000 7fc492a00700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc48400ef10 tx=0x7fc48400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.914+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48400c9c0 con 0x7fc49c06bad0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.914+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc484019070 con 0x7fc49c06bad0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.914+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484014480 con 0x7fc49c06bad0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.914+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19c6e0 con 0x7fc49c06bad0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.914+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c1a89d0 con 0x7fc49c06bad0 2024-04-26T22:01:39.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.915+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c19ca50 con 0x7fc49c06bad0 2024-04-26T22:01:39.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.917+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc484012050 con 0x7fc49c06bad0 2024-04-26T22:01:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.919+0000 7fc498c00700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 0x7fc4800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.919+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc4840d0860 con 0x7fc49c06bad0 2024-04-26T22:01:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.919+0000 7fc49b400700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 0x7fc4800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.920+0000 7fc49b400700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 0x7fc4800a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009fd0 tx=0x7fc48c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:39.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:39.924+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc49c19ca50 con 0x7fc49c06bad0 2024-04-26T22:01:40.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.157+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc49c19cfe0 con 0x7fc4800a3cd0 2024-04-26T22:01:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.170+0000 7fc498c00700 1 -- 172.21.15.12:0/350826432 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340276 (secure 0 0 0) 0x7fc49c19cfe0 con 0x7fc4800a3cd0 2024-04-26T22:01:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.179+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 msgr2=0x7fc4800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.179+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 0x7fc4800a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009fd0 tx=0x7fc48c009260 comp rx=0 tx=0).stop 2024-04-26T22:01:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.179+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 msgr2=0x7fc49c1a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.179+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc48400ef10 tx=0x7fc48400c3a0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 shutdown_connections 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc49c1094a0 0x7fc49c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc4800a3cd0 0x7fc4800a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c06b130 0x7fc49c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 --2- 172.21.15.12:0/350826432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c06bad0 0x7fc49c1a1f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.180+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 >> 172.21.15.12:0/350826432 conn(0x7fc49c0792d0 msgr2=0x7fc49c10e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.181+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 shutdown_connections 2024-04-26T22:01:40.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:40.181+0000 7fc4a2000700 1 -- 172.21.15.12:0/350826432 wait complete. 2024-04-26T22:01:40.183 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T22:01:41.078 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-04-26T22:01:39.171128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647076,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":321,"num_read_kb":269,"num_write":264,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181552,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379940,"kb_used_data":14072,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561884,"statfs":{"total":1151940427776,"available":1151551369216,"internally_reserved":0,"allocated":14409728,"data_stored":7980962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006235"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648136+0000","last_change":"2024-04-26T21:59:44.048850+0000","last_active":"2024-04-26T22:00:44.648136+0000","last_peered":"2024-04-26T22:00:44.648136+0000","last_clean":"2024-04-26T22:00:44.648136+0000","last_became_active":"2024-04-26T21:59:44.048521+0000","last_became_peered":"2024-04-26T21:59:44.048521+0000","last_unstale":"2024-04-26T22:00:44.648136+0000","last_undegraded":"2024-04-26T22:00:44.648136+0000","last_fullsized":"2024-04-26T22:00:44.648136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:09.031610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657383+0000","last_change":"2024-04-26T21:59:44.048900+0000","last_active":"2024-04-26T22:00:44.657383+0000","last_peered":"2024-04-26T22:00:44.657383+0000","last_clean":"2024-04-26T22:00:44.657383+0000","last_became_active":"2024-04-26T21:59:44.048475+0000","last_became_peered":"2024-04-26T21:59:44.048475+0000","last_unstale":"2024-04-26T22:00:44.657383+0000","last_undegraded":"2024-04-26T22:00:44.657383+0000","last_fullsized":"2024-04-26T22:00:44.657383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:01.724256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.660196+0000","last_change":"2024-04-26T21:59:44.045875+0000","last_active":"2024-04-26T22:00:44.660196+0000","last_peered":"2024-04-26T22:00:44.660196+0000","last_clean":"2024-04-26T22:00:44.660196+0000","last_became_active":"2024-04-26T21:59:44.045413+0000","last_became_peered":"2024-04-26T21:59:44.045413+0000","last_unstale":"2024-04-26T22:00:44.660196+0000","last_undegraded":"2024-04-26T22:00:44.660196+0000","last_fullsized":"2024-04-26T22:00:44.660196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:06:19.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.905348+0000","last_change":"2024-04-26T22:00:44.904174+0000","last_active":"2024-04-26T22:00:44.905348+0000","last_peered":"2024-04-26T22:00:44.905348+0000","last_clean":"2024-04-26T22:00:44.905348+0000","last_became_active":"2024-04-26T22:00:44.901768+0000","last_became_peered":"2024-04-26T22:00:44.901768+0000","last_unstale":"2024-04-26T22:00:44.905348+0000","last_undegraded":"2024-04-26T22:00:44.905348+0000","last_fullsized":"2024-04-26T22:00:44.905348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:47.989221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639596+0000","last_change":"2024-04-26T21:59:44.063199+0000","last_active":"2024-04-26T22:00:44.639596+0000","last_peered":"2024-04-26T22:00:44.639596+0000","last_clean":"2024-04-26T22:00:44.639596+0000","last_became_active":"2024-04-26T21:59:44.062845+0000","last_became_peered":"2024-04-26T21:59:44.062845+0000","last_unstale":"2024-04-26T22:00:44.639596+0000","last_undegraded":"2024-04-26T22:00:44.639596+0000","last_fullsized":"2024-04-26T22:00:44.639596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:36.247652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643050+0000","last_change":"2024-04-26T21:59:44.058069+0000","last_active":"2024-04-26T22:00:44.643050+0000","last_peered":"2024-04-26T22:00:44.643050+0000","last_clean":"2024-04-26T22:00:44.643050+0000","last_became_active":"2024-04-26T21:59:44.057726+0000","last_became_peered":"2024-04-26T21:59:44.057726+0000","last_unstale":"2024-04-26T22:00:44.643050+0000","last_undegraded":"2024-04-26T22:00:44.643050+0000","last_fullsized":"2024-04-26T22:00:44.643050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:37.887034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644275+0000","last_change":"2024-04-26T21:59:44.046887+0000","last_active":"2024-04-26T22:00:44.644275+0000","last_peered":"2024-04-26T22:00:44.644275+0000","last_clean":"2024-04-26T22:00:44.644275+0000","last_became_active":"2024-04-26T21:59:44.046507+0000","last_became_peered":"2024-04-26T21:59:44.046507+0000","last_unstale":"2024-04-26T22:00:44.644275+0000","last_undegraded":"2024-04-26T22:00:44.644275+0000","last_fullsized":"2024-04-26T22:00:44.644275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:03.312480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650414+0000","last_change":"2024-04-26T21:59:44.060437+0000","last_active":"2024-04-26T22:00:44.650414+0000","last_peered":"2024-04-26T22:00:44.650414+0000","last_clean":"2024-04-26T22:00:44.650414+0000","last_became_active":"2024-04-26T21:59:44.055341+0000","last_became_peered":"2024-04-26T21:59:44.055341+0000","last_unstale":"2024-04-26T22:00:44.650414+0000","last_undegraded":"2024-04-26T22:00:44.650414+0000","last_fullsized":"2024-04-26T22:00:44.650414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:22.452619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900928+0000","last_change":"2024-04-26T21:59:44.043537+0000","last_active":"2024-04-26T22:00:44.900928+0000","last_peered":"2024-04-26T22:00:44.900928+0000","last_clean":"2024-04-26T22:00:44.900928+0000","last_became_active":"2024-04-26T21:59:44.042227+0000","last_became_peered":"2024-04-26T21:59:44.042227+0000","last_unstale":"2024-04-26T22:00:44.900928+0000","last_undegraded":"2024-04-26T22:00:44.900928+0000","last_fullsized":"2024-04-26T22:00:44.900928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:20.663380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643871+0000","last_change":"2024-04-26T21:59:44.039338+0000","last_active":"2024-04-26T22:00:44.643871+0000","last_peered":"2024-04-26T22:00:44.643871+0000","last_clean":"2024-04-26T22:00:44.643871+0000","last_became_active":"2024-04-26T21:59:44.039000+0000","last_became_peered":"2024-04-26T21:59:44.039000+0000","last_unstale":"2024-04-26T22:00:44.643871+0000","last_undegraded":"2024-04-26T22:00:44.643871+0000","last_fullsized":"2024-04-26T22:00:44.643871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:20.829507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645129+0000","last_change":"2024-04-26T21:59:44.046272+0000","last_active":"2024-04-26T22:00:44.645129+0000","last_peered":"2024-04-26T22:00:44.645129+0000","last_clean":"2024-04-26T22:00:44.645129+0000","last_became_active":"2024-04-26T21:59:44.045235+0000","last_became_peered":"2024-04-26T21:59:44.045235+0000","last_unstale":"2024-04-26T22:00:44.645129+0000","last_undegraded":"2024-04-26T22:00:44.645129+0000","last_fullsized":"2024-04-26T22:00:44.645129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:26.099750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641634+0000","last_change":"2024-04-26T21:59:44.044698+0000","last_active":"2024-04-26T22:00:44.641634+0000","last_peered":"2024-04-26T22:00:44.641634+0000","last_clean":"2024-04-26T22:00:44.641634+0000","last_became_active":"2024-04-26T21:59:44.043394+0000","last_became_peered":"2024-04-26T21:59:44.043394+0000","last_unstale":"2024-04-26T22:00:44.641634+0000","last_undegraded":"2024-04-26T22:00:44.641634+0000","last_fullsized":"2024-04-26T22:00:44.641634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:55.254581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646805+0000","last_change":"2024-04-26T21:59:44.060896+0000","last_active":"2024-04-26T22:00:44.646805+0000","last_peered":"2024-04-26T22:00:44.646805+0000","last_clean":"2024-04-26T22:00:44.646805+0000","last_became_active":"2024-04-26T21:59:44.060010+0000","last_became_peered":"2024-04-26T21:59:44.060010+0000","last_unstale":"2024-04-26T22:00:44.646805+0000","last_undegraded":"2024-04-26T22:00:44.646805+0000","last_fullsized":"2024-04-26T22:00:44.646805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:46.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.661218+0000","last_change":"2024-04-26T21:59:44.050761+0000","last_active":"2024-04-26T22:00:44.661218+0000","last_peered":"2024-04-26T22:00:44.661218+0000","last_clean":"2024-04-26T22:00:44.661218+0000","last_became_active":"2024-04-26T21:59:44.050229+0000","last_became_peered":"2024-04-26T21:59:44.050229+0000","last_unstale":"2024-04-26T22:00:44.661218+0000","last_undegraded":"2024-04-26T22:00:44.661218+0000","last_fullsized":"2024-04-26T22:00:44.661218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:46.114433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.903210+0000","last_change":"2024-04-26T22:00:44.903210+0000","last_active":"2024-04-26T22:00:44.903210+0000","last_peered":"2024-04-26T22:00:44.903210+0000","last_clean":"2024-04-26T22:00:44.903210+0000","last_became_active":"2024-04-26T22:00:44.902517+0000","last_became_peered":"2024-04-26T22:00:44.902517+0000","last_unstale":"2024-04-26T22:00:44.903210+0000","last_undegraded":"2024-04-26T22:00:44.903210+0000","last_fullsized":"2024-04-26T22:00:44.903210+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:44.244533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900249+0000","last_change":"2024-04-26T21:59:44.049616+0000","last_active":"2024-04-26T22:00:44.900249+0000","last_peered":"2024-04-26T22:00:44.900249+0000","last_clean":"2024-04-26T22:00:44.900249+0000","last_became_active":"2024-04-26T21:59:44.049259+0000","last_became_peered":"2024-04-26T21:59:44.049259+0000","last_unstale":"2024-04-26T22:00:44.900249+0000","last_undegraded":"2024-04-26T22:00:44.900249+0000","last_fullsized":"2024-04-26T22:00:44.900249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:25.264219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648002+0000","last_change":"2024-04-26T21:59:44.048657+0000","last_active":"2024-04-26T22:00:44.648002+0000","last_peered":"2024-04-26T22:00:44.648002+0000","last_clean":"2024-04-26T22:00:44.648002+0000","last_became_active":"2024-04-26T21:59:44.048209+0000","last_became_peered":"2024-04-26T21:59:44.048209+0000","last_unstale":"2024-04-26T22:00:44.648002+0000","last_undegraded":"2024-04-26T22:00:44.648002+0000","last_fullsized":"2024-04-26T22:00:44.648002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:01.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647887+0000","last_change":"2024-04-26T21:59:44.058215+0000","last_active":"2024-04-26T22:00:44.647887+0000","last_peered":"2024-04-26T22:00:44.647887+0000","last_clean":"2024-04-26T22:00:44.647887+0000","last_became_active":"2024-04-26T21:59:44.057422+0000","last_became_peered":"2024-04-26T21:59:44.057422+0000","last_unstale":"2024-04-26T22:00:44.647887+0000","last_undegraded":"2024-04-26T22:00:44.647887+0000","last_fullsized":"2024-04-26T22:00:44.647887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:17.215168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900764+0000","last_change":"2024-04-26T21:59:44.054840+0000","last_active":"2024-04-26T22:00:44.900764+0000","last_peered":"2024-04-26T22:00:44.900764+0000","last_clean":"2024-04-26T22:00:44.900764+0000","last_became_active":"2024-04-26T21:59:44.054629+0000","last_became_peered":"2024-04-26T21:59:44.054629+0000","last_unstale":"2024-04-26T22:00:44.900764+0000","last_undegraded":"2024-04-26T22:00:44.900764+0000","last_fullsized":"2024-04-26T22:00:44.900764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:37.204066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900710+0000","last_change":"2024-04-26T21:59:48.145585+0000","last_active":"2024-04-26T22:00:44.900710+0000","last_peered":"2024-04-26T22:00:44.900710+0000","last_clean":"2024-04-26T22:00:44.900710+0000","last_became_active":"2024-04-26T21:59:48.145283+0000","last_became_peered":"2024-04-26T21:59:48.145283+0000","last_unstale":"2024-04-26T22:00:44.900710+0000","last_undegraded":"2024-04-26T22:00:44.900710+0000","last_fullsized":"2024-04-26T22:00:44.900710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:53.096333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.909389+0000","last_change":"2024-04-26T22:00:44.909389+0000","last_active":"2024-04-26T22:00:44.909389+0000","last_peered":"2024-04-26T22:00:44.909389+0000","last_clean":"2024-04-26T22:00:44.909389+0000","last_became_active":"2024-04-26T22:00:44.902131+0000","last_became_peered":"2024-04-26T22:00:44.902131+0000","last_unstale":"2024-04-26T22:00:44.909389+0000","last_undegraded":"2024-04-26T22:00:44.909389+0000","last_fullsized":"2024-04-26T22:00:44.909389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:30.526396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656363+0000","last_change":"2024-04-26T21:59:44.060192+0000","last_active":"2024-04-26T22:00:44.656363+0000","last_peered":"2024-04-26T22:00:44.656363+0000","last_clean":"2024-04-26T22:00:44.656363+0000","last_became_active":"2024-04-26T21:59:44.054969+0000","last_became_peered":"2024-04-26T21:59:44.054969+0000","last_unstale":"2024-04-26T22:00:44.656363+0000","last_undegraded":"2024-04-26T22:00:44.656363+0000","last_fullsized":"2024-04-26T22:00:44.656363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:54.670629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640118+0000","last_change":"2024-04-26T21:59:48.144947+0000","last_active":"2024-04-26T22:00:44.640118+0000","last_peered":"2024-04-26T22:00:44.640118+0000","last_clean":"2024-04-26T22:00:44.640118+0000","last_became_active":"2024-04-26T21:59:48.144547+0000","last_became_peered":"2024-04-26T21:59:48.144547+0000","last_unstale":"2024-04-26T22:00:44.640118+0000","last_undegraded":"2024-04-26T22:00:44.640118+0000","last_fullsized":"2024-04-26T22:00:44.640118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.492136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656735+0000","last_change":"2024-04-26T21:59:43.071253+0000","last_active":"2024-04-26T22:00:44.656735+0000","last_peered":"2024-04-26T22:00:44.656735+0000","last_clean":"2024-04-26T22:00:44.656735+0000","last_became_active":"2024-04-26T21:59:43.070926+0000","last_became_peered":"2024-04-26T21:59:43.070926+0000","last_unstale":"2024-04-26T22:00:44.656735+0000","last_undegraded":"2024-04-26T22:00:44.656735+0000","last_fullsized":"2024-04-26T22:00:44.656735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:30:13.280936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.662577+0000","last_change":"2024-04-26T21:59:44.051951+0000","last_active":"2024-04-26T22:00:44.662577+0000","last_peered":"2024-04-26T22:00:44.662577+0000","last_clean":"2024-04-26T22:00:44.662577+0000","last_became_active":"2024-04-26T21:59:44.051206+0000","last_became_peered":"2024-04-26T21:59:44.051206+0000","last_unstale":"2024-04-26T22:00:44.662577+0000","last_undegraded":"2024-04-26T22:00:44.662577+0000","last_fullsized":"2024-04-26T22:00:44.662577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:05.034106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.659464+0000","last_change":"2024-04-26T22:00:44.659464+0000","last_active":"2024-04-26T22:00:44.659464+0000","last_peered":"2024-04-26T22:00:44.659464+0000","last_clean":"2024-04-26T22:00:44.659464+0000","last_became_active":"2024-04-26T22:00:44.659008+0000","last_became_peered":"2024-04-26T22:00:44.659008+0000","last_unstale":"2024-04-26T22:00:44.659464+0000","last_undegraded":"2024-04-26T22:00:44.659464+0000","last_fullsized":"2024-04-26T22:00:44.659464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:42:42.042268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640094+0000","last_change":"2024-04-26T21:59:43.038322+0000","last_active":"2024-04-26T22:00:44.640094+0000","last_peered":"2024-04-26T22:00:44.640094+0000","last_clean":"2024-04-26T22:00:44.640094+0000","last_became_active":"2024-04-26T21:59:43.037968+0000","last_became_peered":"2024-04-26T21:59:43.037968+0000","last_unstale":"2024-04-26T22:00:44.640094+0000","last_undegraded":"2024-04-26T22:00:44.640094+0000","last_fullsized":"2024-04-26T22:00:44.640094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:21.265367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900539+0000","last_change":"2024-04-26T21:59:44.054970+0000","last_active":"2024-04-26T22:00:44.900539+0000","last_peered":"2024-04-26T22:00:44.900539+0000","last_clean":"2024-04-26T22:00:44.900539+0000","last_became_active":"2024-04-26T21:59:44.054599+0000","last_became_peered":"2024-04-26T21:59:44.054599+0000","last_unstale":"2024-04-26T22:00:44.900539+0000","last_undegraded":"2024-04-26T22:00:44.900539+0000","last_fullsized":"2024-04-26T22:00:44.900539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:22.048544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640401+0000","last_change":"2024-04-26T21:59:48.136704+0000","last_active":"2024-04-26T22:00:44.640401+0000","last_peered":"2024-04-26T22:00:44.640401+0000","last_clean":"2024-04-26T22:00:44.640401+0000","last_became_active":"2024-04-26T21:59:48.136276+0000","last_became_peered":"2024-04-26T21:59:48.136276+0000","last_unstale":"2024-04-26T22:00:44.640401+0000","last_undegraded":"2024-04-26T22:00:44.640401+0000","last_fullsized":"2024-04-26T22:00:44.640401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:05.608497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639812+0000","last_change":"2024-04-26T21:59:43.031556+0000","last_active":"2024-04-26T22:00:44.639812+0000","last_peered":"2024-04-26T22:00:44.639812+0000","last_clean":"2024-04-26T22:00:44.639812+0000","last_became_active":"2024-04-26T21:59:43.031216+0000","last_became_peered":"2024-04-26T21:59:43.031216+0000","last_unstale":"2024-04-26T22:00:44.639812+0000","last_undegraded":"2024-04-26T22:00:44.639812+0000","last_fullsized":"2024-04-26T22:00:44.639812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:51.675630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647111+0000","last_change":"2024-04-26T21:59:44.048291+0000","last_active":"2024-04-26T22:00:44.647111+0000","last_peered":"2024-04-26T22:00:44.647111+0000","last_clean":"2024-04-26T22:00:44.647111+0000","last_became_active":"2024-04-26T21:59:44.047983+0000","last_became_peered":"2024-04-26T21:59:44.047983+0000","last_unstale":"2024-04-26T22:00:44.647111+0000","last_undegraded":"2024-04-26T22:00:44.647111+0000","last_fullsized":"2024-04-26T22:00:44.647111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:17.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900611+0000","last_change":"2024-04-26T21:59:48.129287+0000","last_active":"2024-04-26T22:00:44.900611+0000","last_peered":"2024-04-26T22:00:44.900611+0000","last_clean":"2024-04-26T22:00:44.900611+0000","last_became_active":"2024-04-26T21:59:48.128789+0000","last_became_peered":"2024-04-26T21:59:48.128789+0000","last_unstale":"2024-04-26T22:00:44.900611+0000","last_undegraded":"2024-04-26T22:00:44.900611+0000","last_fullsized":"2024-04-26T22:00:44.900611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:17:46.495215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.649352+0000","last_change":"2024-04-26T22:00:44.649352+0000","last_active":"2024-04-26T22:00:44.649352+0000","last_peered":"2024-04-26T22:00:44.649352+0000","last_clean":"2024-04-26T22:00:44.649352+0000","last_became_active":"2024-04-26T22:00:44.648771+0000","last_became_peered":"2024-04-26T22:00:44.648771+0000","last_unstale":"2024-04-26T22:00:44.649352+0000","last_undegraded":"2024-04-26T22:00:44.649352+0000","last_fullsized":"2024-04-26T22:00:44.649352+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:57.718115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639606+0000","last_change":"2024-04-26T21:59:44.058077+0000","last_active":"2024-04-26T22:00:44.639606+0000","last_peered":"2024-04-26T22:00:44.639606+0000","last_clean":"2024-04-26T22:00:44.639606+0000","last_became_active":"2024-04-26T21:59:44.057720+0000","last_became_peered":"2024-04-26T21:59:44.057720+0000","last_unstale":"2024-04-26T22:00:44.639606+0000","last_undegraded":"2024-04-26T22:00:44.639606+0000","last_fullsized":"2024-04-26T22:00:44.639606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:56.889733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900405+0000","last_change":"2024-04-26T21:59:48.149256+0000","last_active":"2024-04-26T22:00:44.900405+0000","last_peered":"2024-04-26T22:00:44.900405+0000","last_clean":"2024-04-26T22:00:44.900405+0000","last_became_active":"2024-04-26T21:59:48.148832+0000","last_became_peered":"2024-04-26T21:59:48.148832+0000","last_unstale":"2024-04-26T22:00:44.900405+0000","last_undegraded":"2024-04-26T22:00:44.900405+0000","last_fullsized":"2024-04-26T22:00:44.900405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:33:46.120634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.642987+0000","last_change":"2024-04-26T21:59:43.050527+0000","last_active":"2024-04-26T22:00:44.642987+0000","last_peered":"2024-04-26T22:00:44.642987+0000","last_clean":"2024-04-26T22:00:44.642987+0000","last_became_active":"2024-04-26T21:59:43.050168+0000","last_became_peered":"2024-04-26T21:59:43.050168+0000","last_unstale":"2024-04-26T22:00:44.642987+0000","last_undegraded":"2024-04-26T22:00:44.642987+0000","last_fullsized":"2024-04-26T22:00:44.642987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:45.484392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647019+0000","last_change":"2024-04-26T21:59:44.063479+0000","last_active":"2024-04-26T22:00:44.647019+0000","last_peered":"2024-04-26T22:00:44.647019+0000","last_clean":"2024-04-26T22:00:44.647019+0000","last_became_active":"2024-04-26T21:59:44.063205+0000","last_became_peered":"2024-04-26T21:59:44.063205+0000","last_unstale":"2024-04-26T22:00:44.647019+0000","last_undegraded":"2024-04-26T22:00:44.647019+0000","last_fullsized":"2024-04-26T22:00:44.647019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:07.369374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647921+0000","last_change":"2024-04-26T21:59:48.155383+0000","last_active":"2024-04-26T22:00:44.647921+0000","last_peered":"2024-04-26T22:00:44.647921+0000","last_clean":"2024-04-26T22:00:44.647921+0000","last_became_active":"2024-04-26T21:59:48.154948+0000","last_became_peered":"2024-04-26T21:59:48.154948+0000","last_unstale":"2024-04-26T22:00:44.647921+0000","last_undegraded":"2024-04-26T22:00:44.647921+0000","last_fullsized":"2024-04-26T22:00:44.647921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:25.874999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645728+0000","last_change":"2024-04-26T21:59:43.061828+0000","last_active":"2024-04-26T22:00:44.645728+0000","last_peered":"2024-04-26T22:00:44.645728+0000","last_clean":"2024-04-26T22:00:44.645728+0000","last_became_active":"2024-04-26T21:59:43.061440+0000","last_became_peered":"2024-04-26T21:59:43.061440+0000","last_unstale":"2024-04-26T22:00:44.645728+0000","last_undegraded":"2024-04-26T22:00:44.645728+0000","last_fullsized":"2024-04-26T22:00:44.645728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:17.964410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650645+0000","last_change":"2024-04-26T21:59:44.056947+0000","last_active":"2024-04-26T22:00:44.650645+0000","last_peered":"2024-04-26T22:00:44.650645+0000","last_clean":"2024-04-26T22:00:44.650645+0000","last_became_active":"2024-04-26T21:59:44.056537+0000","last_became_peered":"2024-04-26T21:59:44.056537+0000","last_unstale":"2024-04-26T22:00:44.650645+0000","last_undegraded":"2024-04-26T22:00:44.650645+0000","last_fullsized":"2024-04-26T22:00:44.650645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:19.601355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657265+0000","last_change":"2024-04-26T21:59:48.141476+0000","last_active":"2024-04-26T22:00:44.657265+0000","last_peered":"2024-04-26T22:00:44.657265+0000","last_clean":"2024-04-26T22:00:44.657265+0000","last_became_active":"2024-04-26T21:59:48.141028+0000","last_became_peered":"2024-04-26T21:59:48.141028+0000","last_unstale":"2024-04-26T22:00:44.657265+0000","last_undegraded":"2024-04-26T22:00:44.657265+0000","last_fullsized":"2024-04-26T22:00:44.657265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:36.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646537+0000","last_change":"2024-04-26T21:59:43.035587+0000","last_active":"2024-04-26T22:00:44.646537+0000","last_peered":"2024-04-26T22:00:44.646537+0000","last_clean":"2024-04-26T22:00:44.646537+0000","last_became_active":"2024-04-26T21:59:43.035102+0000","last_became_peered":"2024-04-26T21:59:43.035102+0000","last_unstale":"2024-04-26T22:00:44.646537+0000","last_undegraded":"2024-04-26T22:00:44.646537+0000","last_fullsized":"2024-04-26T22:00:44.646537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:52.817321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648389+0000","last_change":"2024-04-26T21:59:44.062002+0000","last_active":"2024-04-26T22:00:44.648389+0000","last_peered":"2024-04-26T22:00:44.648389+0000","last_clean":"2024-04-26T22:00:44.648389+0000","last_became_active":"2024-04-26T21:59:44.058419+0000","last_became_peered":"2024-04-26T21:59:44.058419+0000","last_unstale":"2024-04-26T22:00:44.648389+0000","last_undegraded":"2024-04-26T22:00:44.648389+0000","last_fullsized":"2024-04-26T22:00:44.648389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:48.698093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650871+0000","last_change":"2024-04-26T21:59:48.134540+0000","last_active":"2024-04-26T22:00:44.650871+0000","last_peered":"2024-04-26T22:00:44.650871+0000","last_clean":"2024-04-26T22:00:44.650871+0000","last_became_active":"2024-04-26T21:59:48.133149+0000","last_became_peered":"2024-04-26T21:59:48.133149+0000","last_unstale":"2024-04-26T22:00:44.650871+0000","last_undegraded":"2024-04-26T22:00:44.650871+0000","last_fullsized":"2024-04-26T22:00:44.650871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:18.861332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643614+0000","last_change":"2024-04-26T21:59:43.053360+0000","last_active":"2024-04-26T22:00:44.643614+0000","last_peered":"2024-04-26T22:00:44.643614+0000","last_clean":"2024-04-26T22:00:44.643614+0000","last_became_active":"2024-04-26T21:59:43.053113+0000","last_became_peered":"2024-04-26T21:59:43.053113+0000","last_unstale":"2024-04-26T22:00:44.643614+0000","last_undegraded":"2024-04-26T22:00:44.643614+0000","last_fullsized":"2024-04-26T22:00:44.643614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:38.464573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900806+0000","last_change":"2024-04-26T21:59:44.060319+0000","last_active":"2024-04-26T22:00:44.900806+0000","last_peered":"2024-04-26T22:00:44.900806+0000","last_clean":"2024-04-26T22:00:44.900806+0000","last_became_active":"2024-04-26T21:59:44.059947+0000","last_became_peered":"2024-04-26T21:59:44.059947+0000","last_unstale":"2024-04-26T22:00:44.900806+0000","last_undegraded":"2024-04-26T22:00:44.900806+0000","last_fullsized":"2024-04-26T22:00:44.900806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:01.851669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644179+0000","last_change":"2024-04-26T21:59:48.148358+0000","last_active":"2024-04-26T22:00:44.644179+0000","last_peered":"2024-04-26T22:00:44.644179+0000","last_clean":"2024-04-26T22:00:44.644179+0000","last_became_active":"2024-04-26T21:59:48.143520+0000","last_became_peered":"2024-04-26T21:59:48.143520+0000","last_unstale":"2024-04-26T22:00:44.644179+0000","last_undegraded":"2024-04-26T22:00:44.644179+0000","last_fullsized":"2024-04-26T22:00:44.644179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:23.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645042+0000","last_change":"2024-04-26T21:59:43.037153+0000","last_active":"2024-04-26T22:00:44.645042+0000","last_peered":"2024-04-26T22:00:44.645042+0000","last_clean":"2024-04-26T22:00:44.645042+0000","last_became_active":"2024-04-26T21:59:43.036816+0000","last_became_peered":"2024-04-26T21:59:43.036816+0000","last_unstale":"2024-04-26T22:00:44.645042+0000","last_undegraded":"2024-04-26T22:00:44.645042+0000","last_fullsized":"2024-04-26T22:00:44.645042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:43:44.655304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.662701+0000","last_change":"2024-04-26T21:59:44.047947+0000","last_active":"2024-04-26T22:00:44.662701+0000","last_peered":"2024-04-26T22:00:44.662701+0000","last_clean":"2024-04-26T22:00:44.662701+0000","last_became_active":"2024-04-26T21:59:44.047586+0000","last_became_peered":"2024-04-26T21:59:44.047586+0000","last_unstale":"2024-04-26T22:00:44.662701+0000","last_undegraded":"2024-04-26T22:00:44.662701+0000","last_fullsized":"2024-04-26T22:00:44.662701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:49.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641250+0000","last_change":"2024-04-26T21:59:48.150787+0000","last_active":"2024-04-26T22:00:44.641250+0000","last_peered":"2024-04-26T22:00:44.641250+0000","last_clean":"2024-04-26T22:00:44.641250+0000","last_became_active":"2024-04-26T21:59:48.150004+0000","last_became_peered":"2024-04-26T21:59:48.150004+0000","last_unstale":"2024-04-26T22:00:44.641250+0000","last_undegraded":"2024-04-26T22:00:44.641250+0000","last_fullsized":"2024-04-26T22:00:44.641250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:19.657837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646863+0000","last_change":"2024-04-26T21:59:43.050931+0000","last_active":"2024-04-26T22:00:44.646863+0000","last_peered":"2024-04-26T22:00:44.646863+0000","last_clean":"2024-04-26T22:00:44.646863+0000","last_became_active":"2024-04-26T21:59:43.050508+0000","last_became_peered":"2024-04-26T21:59:43.050508+0000","last_unstale":"2024-04-26T22:00:44.646863+0000","last_undegraded":"2024-04-26T22:00:44.646863+0000","last_fullsized":"2024-04-26T22:00:44.646863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:08.378649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900623+0000","last_change":"2024-04-26T21:59:44.039078+0000","last_active":"2024-04-26T22:00:44.900623+0000","last_peered":"2024-04-26T22:00:44.900623+0000","last_clean":"2024-04-26T22:00:44.900623+0000","last_became_active":"2024-04-26T21:59:44.038734+0000","last_became_peered":"2024-04-26T21:59:44.038734+0000","last_unstale":"2024-04-26T22:00:44.900623+0000","last_undegraded":"2024-04-26T22:00:44.900623+0000","last_fullsized":"2024-04-26T22:00:44.900623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:07.551134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644453+0000","last_change":"2024-04-26T21:59:48.150464+0000","last_active":"2024-04-26T22:00:44.644453+0000","last_peered":"2024-04-26T22:00:44.644453+0000","last_clean":"2024-04-26T22:00:44.644453+0000","last_became_active":"2024-04-26T21:59:48.150095+0000","last_became_peered":"2024-04-26T21:59:48.150095+0000","last_unstale":"2024-04-26T22:00:44.644453+0000","last_undegraded":"2024-04-26T22:00:44.644453+0000","last_fullsized":"2024-04-26T22:00:44.644453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:53.152840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":234,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:01:34.756692+0000","last_change":"2024-04-26T21:59:43.038751+0000","last_active":"2024-04-26T22:01:34.756692+0000","last_peered":"2024-04-26T22:01:34.756692+0000","last_clean":"2024-04-26T22:01:34.756692+0000","last_became_active":"2024-04-26T21:59:43.038337+0000","last_became_peered":"2024-04-26T21:59:43.038337+0000","last_unstale":"2024-04-26T22:01:34.756692+0000","last_undegraded":"2024-04-26T22:01:34.756692+0000","last_fullsized":"2024-04-26T22:01:34.756692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.830357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55676,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":179,"num_read_kb":144,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639924+0000","last_change":"2024-04-26T21:59:44.040170+0000","last_active":"2024-04-26T22:00:44.639924+0000","last_peered":"2024-04-26T22:00:44.639924+0000","last_clean":"2024-04-26T22:00:44.639924+0000","last_became_active":"2024-04-26T21:59:44.039713+0000","last_became_peered":"2024-04-26T21:59:44.039713+0000","last_unstale":"2024-04-26T22:00:44.639924+0000","last_undegraded":"2024-04-26T22:00:44.639924+0000","last_fullsized":"2024-04-26T22:00:44.639924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:49.036612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.665987+0000","last_change":"2024-04-26T22:00:44.665631+0000","last_active":"2024-04-26T22:00:44.665987+0000","last_peered":"2024-04-26T22:00:44.665987+0000","last_clean":"2024-04-26T22:00:44.665987+0000","last_became_active":"2024-04-26T22:00:44.665350+0000","last_became_peered":"2024-04-26T22:00:44.665350+0000","last_unstale":"2024-04-26T22:00:44.665987+0000","last_undegraded":"2024-04-26T22:00:44.665987+0000","last_fullsized":"2024-04-26T22:00:44.665987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:54.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900501+0000","last_change":"2024-04-26T21:59:43.039994+0000","last_active":"2024-04-26T22:00:44.900501+0000","last_peered":"2024-04-26T22:00:44.900501+0000","last_clean":"2024-04-26T22:00:44.900501+0000","last_became_active":"2024-04-26T21:59:43.039671+0000","last_became_peered":"2024-04-26T21:59:43.039671+0000","last_unstale":"2024-04-26T22:00:44.900501+0000","last_undegraded":"2024-04-26T22:00:44.900501+0000","last_fullsized":"2024-04-26T22:00:44.900501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:59.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646609+0000","last_change":"2024-04-26T21:59:44.065515+0000","last_active":"2024-04-26T22:00:44.646609+0000","last_peered":"2024-04-26T22:00:44.646609+0000","last_clean":"2024-04-26T22:00:44.646609+0000","last_became_active":"2024-04-26T21:59:44.065120+0000","last_became_peered":"2024-04-26T21:59:44.065120+0000","last_unstale":"2024-04-26T22:00:44.646609+0000","last_undegraded":"2024-04-26T22:00:44.646609+0000","last_fullsized":"2024-04-26T22:00:44.646609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:09.709198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647004+0000","last_change":"2024-04-26T21:59:48.149372+0000","last_active":"2024-04-26T22:00:44.647004+0000","last_peered":"2024-04-26T22:00:44.647004+0000","last_clean":"2024-04-26T22:00:44.647004+0000","last_became_active":"2024-04-26T21:59:48.148920+0000","last_became_peered":"2024-04-26T21:59:48.148920+0000","last_unstale":"2024-04-26T22:00:44.647004+0000","last_undegraded":"2024-04-26T22:00:44.647004+0000","last_fullsized":"2024-04-26T22:00:44.647004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:46.303729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647893+0000","last_change":"2024-04-26T21:59:43.040576+0000","last_active":"2024-04-26T22:00:44.647893+0000","last_peered":"2024-04-26T22:00:44.647893+0000","last_clean":"2024-04-26T22:00:44.647893+0000","last_became_active":"2024-04-26T21:59:43.040217+0000","last_became_peered":"2024-04-26T21:59:43.040217+0000","last_unstale":"2024-04-26T22:00:44.647893+0000","last_undegraded":"2024-04-26T22:00:44.647893+0000","last_fullsized":"2024-04-26T22:00:44.647893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:58.860607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644712+0000","last_change":"2024-04-26T21:59:44.067746+0000","last_active":"2024-04-26T22:00:44.644712+0000","last_peered":"2024-04-26T22:00:44.644712+0000","last_clean":"2024-04-26T22:00:44.644712+0000","last_became_active":"2024-04-26T21:59:44.067434+0000","last_became_peered":"2024-04-26T21:59:44.067434+0000","last_unstale":"2024-04-26T22:00:44.644712+0000","last_undegraded":"2024-04-26T22:00:44.644712+0000","last_fullsized":"2024-04-26T22:00:44.644712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:45.304899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647057+0000","last_change":"2024-04-26T21:59:48.157380+0000","last_active":"2024-04-26T22:00:44.647057+0000","last_peered":"2024-04-26T22:00:44.647057+0000","last_clean":"2024-04-26T22:00:44.647057+0000","last_became_active":"2024-04-26T21:59:48.155887+0000","last_became_peered":"2024-04-26T21:59:48.155887+0000","last_unstale":"2024-04-26T22:00:44.647057+0000","last_undegraded":"2024-04-26T22:00:44.647057+0000","last_fullsized":"2024-04-26T22:00:44.647057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.339191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640385+0000","last_change":"2024-04-26T21:59:43.036773+0000","last_active":"2024-04-26T22:00:44.640385+0000","last_peered":"2024-04-26T22:00:44.640385+0000","last_clean":"2024-04-26T22:00:44.640385+0000","last_became_active":"2024-04-26T21:59:43.036384+0000","last_became_peered":"2024-04-26T21:59:43.036384+0000","last_unstale":"2024-04-26T22:00:44.640385+0000","last_undegraded":"2024-04-26T22:00:44.640385+0000","last_fullsized":"2024-04-26T22:00:44.640385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:33.841309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.661062+0000","last_change":"2024-04-26T22:00:44.661062+0000","last_active":"2024-04-26T22:00:44.661062+0000","last_peered":"2024-04-26T22:00:44.661062+0000","last_clean":"2024-04-26T22:00:44.661062+0000","last_became_active":"2024-04-26T22:00:44.660471+0000","last_became_peered":"2024-04-26T22:00:44.660471+0000","last_unstale":"2024-04-26T22:00:44.661062+0000","last_undegraded":"2024-04-26T22:00:44.661062+0000","last_fullsized":"2024-04-26T22:00:44.661062+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:30.077158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644190+0000","last_change":"2024-04-26T21:59:48.156690+0000","last_active":"2024-04-26T22:00:44.644190+0000","last_peered":"2024-04-26T22:00:44.644190+0000","last_clean":"2024-04-26T22:00:44.644190+0000","last_became_active":"2024-04-26T21:59:48.155067+0000","last_became_peered":"2024-04-26T21:59:48.155067+0000","last_unstale":"2024-04-26T22:00:44.644190+0000","last_undegraded":"2024-04-26T22:00:44.644190+0000","last_fullsized":"2024-04-26T22:00:44.644190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:39.366540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639430+0000","last_change":"2024-04-26T21:59:43.050964+0000","last_active":"2024-04-26T22:00:44.639430+0000","last_peered":"2024-04-26T22:00:44.639430+0000","last_clean":"2024-04-26T22:00:44.639430+0000","last_became_active":"2024-04-26T21:59:43.050622+0000","last_became_peered":"2024-04-26T21:59:43.050622+0000","last_unstale":"2024-04-26T22:00:44.639430+0000","last_undegraded":"2024-04-26T22:00:44.639430+0000","last_fullsized":"2024-04-26T22:00:44.639430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:18.246708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646546+0000","last_change":"2024-04-26T21:59:44.044126+0000","last_active":"2024-04-26T22:00:44.646546+0000","last_peered":"2024-04-26T22:00:44.646546+0000","last_clean":"2024-04-26T22:00:44.646546+0000","last_became_active":"2024-04-26T21:59:44.043800+0000","last_became_peered":"2024-04-26T21:59:44.043800+0000","last_unstale":"2024-04-26T22:00:44.646546+0000","last_undegraded":"2024-04-26T22:00:44.646546+0000","last_fullsized":"2024-04-26T22:00:44.646546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:14:46.231703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646388+0000","last_change":"2024-04-26T21:59:48.134868+0000","last_active":"2024-04-26T22:00:44.646388+0000","last_peered":"2024-04-26T22:00:44.646388+0000","last_clean":"2024-04-26T22:00:44.646388+0000","last_became_active":"2024-04-26T21:59:48.134545+0000","last_became_peered":"2024-04-26T21:59:48.134545+0000","last_unstale":"2024-04-26T22:00:44.646388+0000","last_undegraded":"2024-04-26T22:00:44.646388+0000","last_fullsized":"2024-04-26T22:00:44.646388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:59.452010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640177+0000","last_change":"2024-04-26T21:59:43.038912+0000","last_active":"2024-04-26T22:00:44.640177+0000","last_peered":"2024-04-26T22:00:44.640177+0000","last_clean":"2024-04-26T22:00:44.640177+0000","last_became_active":"2024-04-26T21:59:43.038597+0000","last_became_peered":"2024-04-26T21:59:43.038597+0000","last_unstale":"2024-04-26T22:00:44.640177+0000","last_undegraded":"2024-04-26T22:00:44.640177+0000","last_fullsized":"2024-04-26T22:00:44.640177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:37.087595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648113+0000","last_change":"2024-04-26T21:59:44.043297+0000","last_active":"2024-04-26T22:00:44.648113+0000","last_peered":"2024-04-26T22:00:44.648113+0000","last_clean":"2024-04-26T22:00:44.648113+0000","last_became_active":"2024-04-26T21:59:44.042952+0000","last_became_peered":"2024-04-26T21:59:44.042952+0000","last_unstale":"2024-04-26T22:00:44.648113+0000","last_undegraded":"2024-04-26T22:00:44.648113+0000","last_fullsized":"2024-04-26T22:00:44.648113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:18.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639597+0000","last_change":"2024-04-26T21:59:48.155119+0000","last_active":"2024-04-26T22:00:44.639597+0000","last_peered":"2024-04-26T22:00:44.639597+0000","last_clean":"2024-04-26T22:00:44.639597+0000","last_became_active":"2024-04-26T21:59:48.154593+0000","last_became_peered":"2024-04-26T21:59:48.154593+0000","last_unstale":"2024-04-26T22:00:44.639597+0000","last_undegraded":"2024-04-26T22:00:44.639597+0000","last_fullsized":"2024-04-26T22:00:44.639597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:01.896610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646386+0000","last_change":"2024-04-26T21:59:43.067817+0000","last_active":"2024-04-26T22:00:44.646386+0000","last_peered":"2024-04-26T22:00:44.646386+0000","last_clean":"2024-04-26T22:00:44.646386+0000","last_became_active":"2024-04-26T21:59:43.067450+0000","last_became_peered":"2024-04-26T21:59:43.067450+0000","last_unstale":"2024-04-26T22:00:44.646386+0000","last_undegraded":"2024-04-26T22:00:44.646386+0000","last_fullsized":"2024-04-26T22:00:44.646386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:29.425903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643412+0000","last_change":"2024-04-26T21:59:44.055085+0000","last_active":"2024-04-26T22:00:44.643412+0000","last_peered":"2024-04-26T22:00:44.643412+0000","last_clean":"2024-04-26T22:00:44.643412+0000","last_became_active":"2024-04-26T21:59:44.054627+0000","last_became_peered":"2024-04-26T21:59:44.054627+0000","last_unstale":"2024-04-26T22:00:44.643412+0000","last_undegraded":"2024-04-26T22:00:44.643412+0000","last_fullsized":"2024-04-26T22:00:44.643412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:12.234287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645845+0000","last_change":"2024-04-26T21:59:48.147333+0000","last_active":"2024-04-26T22:00:44.645845+0000","last_peered":"2024-04-26T22:00:44.645845+0000","last_clean":"2024-04-26T22:00:44.645845+0000","last_became_active":"2024-04-26T21:59:48.146926+0000","last_became_peered":"2024-04-26T21:59:48.146926+0000","last_unstale":"2024-04-26T22:00:44.645845+0000","last_undegraded":"2024-04-26T22:00:44.645845+0000","last_fullsized":"2024-04-26T22:00:44.645845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644463+0000","last_change":"2024-04-26T21:59:43.050434+0000","last_active":"2024-04-26T22:00:44.644463+0000","last_peered":"2024-04-26T22:00:44.644463+0000","last_clean":"2024-04-26T22:00:44.644463+0000","last_became_active":"2024-04-26T21:59:43.050096+0000","last_became_peered":"2024-04-26T21:59:43.050096+0000","last_unstale":"2024-04-26T22:00:44.644463+0000","last_undegraded":"2024-04-26T22:00:44.644463+0000","last_fullsized":"2024-04-26T22:00:44.644463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:22.769674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643170+0000","last_change":"2024-04-26T21:59:44.048302+0000","last_active":"2024-04-26T22:00:44.643170+0000","last_peered":"2024-04-26T22:00:44.643170+0000","last_clean":"2024-04-26T22:00:44.643170+0000","last_became_active":"2024-04-26T21:59:44.047898+0000","last_became_peered":"2024-04-26T21:59:44.047898+0000","last_unstale":"2024-04-26T22:00:44.643170+0000","last_undegraded":"2024-04-26T22:00:44.643170+0000","last_fullsized":"2024-04-26T22:00:44.643170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:55.718558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646834+0000","last_change":"2024-04-26T21:59:48.148025+0000","last_active":"2024-04-26T22:00:44.646834+0000","last_peered":"2024-04-26T22:00:44.646834+0000","last_clean":"2024-04-26T22:00:44.646834+0000","last_became_active":"2024-04-26T21:59:48.147065+0000","last_became_peered":"2024-04-26T21:59:48.147065+0000","last_unstale":"2024-04-26T22:00:44.646834+0000","last_undegraded":"2024-04-26T22:00:44.646834+0000","last_fullsized":"2024-04-26T22:00:44.646834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:23.932501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646753+0000","last_change":"2024-04-26T21:59:44.045186+0000","last_active":"2024-04-26T22:00:44.646753+0000","last_peered":"2024-04-26T22:00:44.646753+0000","last_clean":"2024-04-26T22:00:44.646753+0000","last_became_active":"2024-04-26T21:59:44.044761+0000","last_became_peered":"2024-04-26T21:59:44.044761+0000","last_unstale":"2024-04-26T22:00:44.646753+0000","last_undegraded":"2024-04-26T22:00:44.646753+0000","last_fullsized":"2024-04-26T22:00:44.646753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:32.744412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645421+0000","last_change":"2024-04-26T21:59:43.068695+0000","last_active":"2024-04-26T22:00:44.645421+0000","last_peered":"2024-04-26T22:00:44.645421+0000","last_clean":"2024-04-26T22:00:44.645421+0000","last_became_active":"2024-04-26T21:59:43.068210+0000","last_became_peered":"2024-04-26T21:59:43.068210+0000","last_unstale":"2024-04-26T22:00:44.645421+0000","last_undegraded":"2024-04-26T22:00:44.645421+0000","last_fullsized":"2024-04-26T22:00:44.645421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:19.914872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.663734+0000","last_change":"2024-04-26T22:00:44.663734+0000","last_active":"2024-04-26T22:00:44.663734+0000","last_peered":"2024-04-26T22:00:44.663734+0000","last_clean":"2024-04-26T22:00:44.663734+0000","last_became_active":"2024-04-26T22:00:44.649296+0000","last_became_peered":"2024-04-26T22:00:44.649296+0000","last_unstale":"2024-04-26T22:00:44.663734+0000","last_undegraded":"2024-04-26T22:00:44.663734+0000","last_fullsized":"2024-04-26T22:00:44.663734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:30.645448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651556+0000","last_change":"2024-04-26T21:59:44.052732+0000","last_active":"2024-04-26T22:00:44.651556+0000","last_peered":"2024-04-26T22:00:44.651556+0000","last_clean":"2024-04-26T22:00:44.651556+0000","last_became_active":"2024-04-26T21:59:44.051202+0000","last_became_peered":"2024-04-26T21:59:44.051202+0000","last_unstale":"2024-04-26T22:00:44.651556+0000","last_undegraded":"2024-04-26T22:00:44.651556+0000","last_fullsized":"2024-04-26T22:00:44.651556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:20.887239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641223+0000","last_change":"2024-04-26T21:59:43.054776+0000","last_active":"2024-04-26T22:00:44.641223+0000","last_peered":"2024-04-26T22:00:44.641223+0000","last_clean":"2024-04-26T22:00:44.641223+0000","last_became_active":"2024-04-26T21:59:43.054410+0000","last_became_peered":"2024-04-26T21:59:43.054410+0000","last_unstale":"2024-04-26T22:00:44.641223+0000","last_undegraded":"2024-04-26T22:00:44.641223+0000","last_fullsized":"2024-04-26T22:00:44.641223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:49:11.675134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644238+0000","last_change":"2024-04-26T21:58:22.546842+0000","last_active":"2024-04-26T22:00:44.644238+0000","last_peered":"2024-04-26T22:00:44.644238+0000","last_clean":"2024-04-26T22:00:44.644238+0000","last_became_active":"2024-04-26T21:58:22.546448+0000","last_became_peered":"2024-04-26T21:58:22.546448+0000","last_unstale":"2024-04-26T22:00:44.644238+0000","last_undegraded":"2024-04-26T22:00:44.644238+0000","last_fullsized":"2024-04-26T22:00:44.644238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:54:29.502062+0000","last_clean_scrub_stamp":"2024-04-26T21:54:29.502062+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:31.851240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646875+0000","last_change":"2024-04-26T21:59:48.152986+0000","last_active":"2024-04-26T22:00:44.646875+0000","last_peered":"2024-04-26T22:00:44.646875+0000","last_clean":"2024-04-26T22:00:44.646875+0000","last_became_active":"2024-04-26T21:59:48.152606+0000","last_became_peered":"2024-04-26T21:59:48.152606+0000","last_unstale":"2024-04-26T22:00:44.646875+0000","last_undegraded":"2024-04-26T22:00:44.646875+0000","last_fullsized":"2024-04-26T22:00:44.646875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645256+0000","last_change":"2024-04-26T21:59:44.060086+0000","last_active":"2024-04-26T22:00:44.645256+0000","last_peered":"2024-04-26T22:00:44.645256+0000","last_clean":"2024-04-26T22:00:44.645256+0000","last_became_active":"2024-04-26T21:59:44.059742+0000","last_became_peered":"2024-04-26T21:59:44.059742+0000","last_unstale":"2024-04-26T22:00:44.645256+0000","last_undegraded":"2024-04-26T22:00:44.645256+0000","last_fullsized":"2024-04-26T22:00:44.645256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.678769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644321+0000","last_change":"2024-04-26T21:59:43.034726+0000","last_active":"2024-04-26T22:00:44.644321+0000","last_peered":"2024-04-26T22:00:44.644321+0000","last_clean":"2024-04-26T22:00:44.644321+0000","last_became_active":"2024-04-26T21:59:43.034407+0000","last_became_peered":"2024-04-26T21:59:43.034407+0000","last_unstale":"2024-04-26T22:00:44.644321+0000","last_undegraded":"2024-04-26T22:00:44.644321+0000","last_fullsized":"2024-04-26T22:00:44.644321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:37.722860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647601+0000","last_change":"2024-04-26T21:59:48.136044+0000","last_active":"2024-04-26T22:00:44.647601+0000","last_peered":"2024-04-26T22:00:44.647601+0000","last_clean":"2024-04-26T22:00:44.647601+0000","last_became_active":"2024-04-26T21:59:48.135619+0000","last_became_peered":"2024-04-26T21:59:48.135619+0000","last_unstale":"2024-04-26T22:00:44.647601+0000","last_undegraded":"2024-04-26T22:00:44.647601+0000","last_fullsized":"2024-04-26T22:00:44.647601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:24.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639995+0000","last_change":"2024-04-26T21:59:43.059149+0000","last_active":"2024-04-26T22:00:44.639995+0000","last_peered":"2024-04-26T22:00:44.639995+0000","last_clean":"2024-04-26T22:00:44.639995+0000","last_became_active":"2024-04-26T21:59:43.058816+0000","last_became_peered":"2024-04-26T21:59:43.058816+0000","last_unstale":"2024-04-26T22:00:44.639995+0000","last_undegraded":"2024-04-26T22:00:44.639995+0000","last_fullsized":"2024-04-26T22:00:44.639995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:04.441843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646064+0000","last_change":"2024-04-26T21:59:44.063544+0000","last_active":"2024-04-26T22:00:44.646064+0000","last_peered":"2024-04-26T22:00:44.646064+0000","last_clean":"2024-04-26T22:00:44.646064+0000","last_became_active":"2024-04-26T21:59:44.063179+0000","last_became_peered":"2024-04-26T21:59:44.063179+0000","last_unstale":"2024-04-26T22:00:44.646064+0000","last_undegraded":"2024-04-26T22:00:44.646064+0000","last_fullsized":"2024-04-26T22:00:44.646064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:44.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.650531+0000","last_change":"2024-04-26T21:59:48.131695+0000","last_active":"2024-04-26T22:00:44.650531+0000","last_peered":"2024-04-26T22:00:44.650531+0000","last_clean":"2024-04-26T22:00:44.650531+0000","last_became_active":"2024-04-26T21:59:48.131246+0000","last_became_peered":"2024-04-26T21:59:48.131246+0000","last_unstale":"2024-04-26T22:00:44.650531+0000","last_undegraded":"2024-04-26T22:00:44.650531+0000","last_fullsized":"2024-04-26T22:00:44.650531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:53.128553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647819+0000","last_change":"2024-04-26T21:59:43.067463+0000","last_active":"2024-04-26T22:00:44.647819+0000","last_peered":"2024-04-26T22:00:44.647819+0000","last_clean":"2024-04-26T22:00:44.647819+0000","last_became_active":"2024-04-26T21:59:43.067009+0000","last_became_peered":"2024-04-26T21:59:43.067009+0000","last_unstale":"2024-04-26T22:00:44.647819+0000","last_undegraded":"2024-04-26T22:00:44.647819+0000","last_fullsized":"2024-04-26T22:00:44.647819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:35.728070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.658109+0000","last_change":"2024-04-26T21:59:44.054781+0000","last_active":"2024-04-26T22:00:44.658109+0000","last_peered":"2024-04-26T22:00:44.658109+0000","last_clean":"2024-04-26T22:00:44.658109+0000","last_became_active":"2024-04-26T21:59:44.049439+0000","last_became_peered":"2024-04-26T21:59:44.049439+0000","last_unstale":"2024-04-26T22:00:44.658109+0000","last_undegraded":"2024-04-26T22:00:44.658109+0000","last_fullsized":"2024-04-26T22:00:44.658109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:50.523835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900373+0000","last_change":"2024-04-26T21:59:48.127776+0000","last_active":"2024-04-26T22:00:44.900373+0000","last_peered":"2024-04-26T22:00:44.900373+0000","last_clean":"2024-04-26T22:00:44.900373+0000","last_became_active":"2024-04-26T21:59:48.127207+0000","last_became_peered":"2024-04-26T21:59:48.127207+0000","last_unstale":"2024-04-26T22:00:44.900373+0000","last_undegraded":"2024-04-26T22:00:44.900373+0000","last_fullsized":"2024-04-26T22:00:44.900373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:23.794371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.641087+0000","last_change":"2024-04-26T21:59:43.050320+0000","last_active":"2024-04-26T22:00:44.641087+0000","last_peered":"2024-04-26T22:00:44.641087+0000","last_clean":"2024-04-26T22:00:44.641087+0000","last_became_active":"2024-04-26T21:59:43.049890+0000","last_became_peered":"2024-04-26T21:59:43.049890+0000","last_unstale":"2024-04-26T22:00:44.641087+0000","last_undegraded":"2024-04-26T22:00:44.641087+0000","last_fullsized":"2024-04-26T22:00:44.641087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:53.589776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.657485+0000","last_change":"2024-04-26T21:59:44.057851+0000","last_active":"2024-04-26T22:00:44.657485+0000","last_peered":"2024-04-26T22:00:44.657485+0000","last_clean":"2024-04-26T22:00:44.657485+0000","last_became_active":"2024-04-26T21:59:44.057525+0000","last_became_peered":"2024-04-26T21:59:44.057525+0000","last_unstale":"2024-04-26T22:00:44.657485+0000","last_undegraded":"2024-04-26T22:00:44.657485+0000","last_fullsized":"2024-04-26T22:00:44.657485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:30.986251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640401+0000","last_change":"2024-04-26T21:59:48.154962+0000","last_active":"2024-04-26T22:00:44.640401+0000","last_peered":"2024-04-26T22:00:44.640401+0000","last_clean":"2024-04-26T22:00:44.640401+0000","last_became_active":"2024-04-26T21:59:48.154479+0000","last_became_peered":"2024-04-26T21:59:48.154479+0000","last_unstale":"2024-04-26T22:00:44.640401+0000","last_undegraded":"2024-04-26T22:00:44.640401+0000","last_fullsized":"2024-04-26T22:00:44.640401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:30.415432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900323+0000","last_change":"2024-04-26T21:59:43.051186+0000","last_active":"2024-04-26T22:00:44.900323+0000","last_peered":"2024-04-26T22:00:44.900323+0000","last_clean":"2024-04-26T22:00:44.900323+0000","last_became_active":"2024-04-26T21:59:43.050768+0000","last_became_peered":"2024-04-26T21:59:43.050768+0000","last_unstale":"2024-04-26T22:00:44.900323+0000","last_undegraded":"2024-04-26T22:00:44.900323+0000","last_fullsized":"2024-04-26T22:00:44.900323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.237229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639400+0000","last_change":"2024-04-26T21:59:44.039113+0000","last_active":"2024-04-26T22:00:44.639400+0000","last_peered":"2024-04-26T22:00:44.639400+0000","last_clean":"2024-04-26T22:00:44.639400+0000","last_became_active":"2024-04-26T21:59:44.038610+0000","last_became_peered":"2024-04-26T21:59:44.038610+0000","last_unstale":"2024-04-26T22:00:44.639400+0000","last_undegraded":"2024-04-26T22:00:44.639400+0000","last_fullsized":"2024-04-26T22:00:44.639400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:49:39.450603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647252+0000","last_change":"2024-04-26T21:59:48.157058+0000","last_active":"2024-04-26T22:00:44.647252+0000","last_peered":"2024-04-26T22:00:44.647252+0000","last_clean":"2024-04-26T22:00:44.647252+0000","last_became_active":"2024-04-26T21:59:48.153897+0000","last_became_peered":"2024-04-26T21:59:48.153897+0000","last_unstale":"2024-04-26T22:00:44.647252+0000","last_undegraded":"2024-04-26T22:00:44.647252+0000","last_fullsized":"2024-04-26T22:00:44.647252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:34:08.044493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651113+0000","last_change":"2024-04-26T21:59:43.061101+0000","last_active":"2024-04-26T22:00:44.651113+0000","last_peered":"2024-04-26T22:00:44.651113+0000","last_clean":"2024-04-26T22:00:44.651113+0000","last_became_active":"2024-04-26T21:59:43.060762+0000","last_became_peered":"2024-04-26T21:59:43.060762+0000","last_unstale":"2024-04-26T22:00:44.651113+0000","last_undegraded":"2024-04-26T22:00:44.651113+0000","last_fullsized":"2024-04-26T22:00:44.651113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:07.961535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647073+0000","last_change":"2024-04-26T21:59:44.047437+0000","last_active":"2024-04-26T22:00:44.647073+0000","last_peered":"2024-04-26T22:00:44.647073+0000","last_clean":"2024-04-26T22:00:44.647073+0000","last_became_active":"2024-04-26T21:59:44.047127+0000","last_became_peered":"2024-04-26T21:59:44.047127+0000","last_unstale":"2024-04-26T22:00:44.647073+0000","last_undegraded":"2024-04-26T22:00:44.647073+0000","last_fullsized":"2024-04-26T22:00:44.647073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:12.259965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.660193+0000","last_change":"2024-04-26T22:00:44.660193+0000","last_active":"2024-04-26T22:00:44.660193+0000","last_peered":"2024-04-26T22:00:44.660193+0000","last_clean":"2024-04-26T22:00:44.660193+0000","last_became_active":"2024-04-26T22:00:44.645540+0000","last_became_peered":"2024-04-26T22:00:44.645540+0000","last_unstale":"2024-04-26T22:00:44.660193+0000","last_undegraded":"2024-04-26T22:00:44.660193+0000","last_fullsized":"2024-04-26T22:00:44.660193+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:03.437665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900467+0000","last_change":"2024-04-26T21:59:43.043901+0000","last_active":"2024-04-26T22:00:44.900467+0000","last_peered":"2024-04-26T22:00:44.900467+0000","last_clean":"2024-04-26T22:00:44.900467+0000","last_became_active":"2024-04-26T21:59:43.043572+0000","last_became_peered":"2024-04-26T21:59:43.043572+0000","last_unstale":"2024-04-26T22:00:44.900467+0000","last_undegraded":"2024-04-26T22:00:44.900467+0000","last_fullsized":"2024-04-26T22:00:44.900467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.107295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.656522+0000","last_change":"2024-04-26T21:59:44.059451+0000","last_active":"2024-04-26T22:00:44.656522+0000","last_peered":"2024-04-26T22:00:44.656522+0000","last_clean":"2024-04-26T22:00:44.656522+0000","last_became_active":"2024-04-26T21:59:44.056771+0000","last_became_peered":"2024-04-26T21:59:44.056771+0000","last_unstale":"2024-04-26T22:00:44.656522+0000","last_undegraded":"2024-04-26T22:00:44.656522+0000","last_fullsized":"2024-04-26T22:00:44.656522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:53.641182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.640389+0000","last_change":"2024-04-26T21:59:48.130222+0000","last_active":"2024-04-26T22:00:44.640389+0000","last_peered":"2024-04-26T22:00:44.640389+0000","last_clean":"2024-04-26T22:00:44.640389+0000","last_became_active":"2024-04-26T21:59:48.129379+0000","last_became_peered":"2024-04-26T21:59:48.129379+0000","last_unstale":"2024-04-26T22:00:44.640389+0000","last_undegraded":"2024-04-26T22:00:44.640389+0000","last_fullsized":"2024-04-26T22:00:44.640389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:26.152253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.639569+0000","last_change":"2024-04-26T21:59:43.035503+0000","last_active":"2024-04-26T22:00:44.639569+0000","last_peered":"2024-04-26T22:00:44.639569+0000","last_clean":"2024-04-26T22:00:44.639569+0000","last_became_active":"2024-04-26T21:59:43.035180+0000","last_became_peered":"2024-04-26T21:59:43.035180+0000","last_unstale":"2024-04-26T22:00:44.639569+0000","last_undegraded":"2024-04-26T22:00:44.639569+0000","last_fullsized":"2024-04-26T22:00:44.639569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:45.696707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644574+0000","last_change":"2024-04-26T21:59:44.054444+0000","last_active":"2024-04-26T22:00:44.644574+0000","last_peered":"2024-04-26T22:00:44.644574+0000","last_clean":"2024-04-26T22:00:44.644574+0000","last_became_active":"2024-04-26T21:59:44.054035+0000","last_became_peered":"2024-04-26T21:59:44.054035+0000","last_unstale":"2024-04-26T22:00:44.644574+0000","last_undegraded":"2024-04-26T22:00:44.644574+0000","last_fullsized":"2024-04-26T22:00:44.644574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:33:04.905286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900354+0000","last_change":"2024-04-26T21:59:48.141551+0000","last_active":"2024-04-26T22:00:44.900354+0000","last_peered":"2024-04-26T22:00:44.900354+0000","last_clean":"2024-04-26T22:00:44.900354+0000","last_became_active":"2024-04-26T21:59:48.141166+0000","last_became_peered":"2024-04-26T21:59:48.141166+0000","last_unstale":"2024-04-26T22:00:44.900354+0000","last_undegraded":"2024-04-26T22:00:44.900354+0000","last_fullsized":"2024-04-26T22:00:44.900354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:38.810387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900537+0000","last_change":"2024-04-26T21:59:43.059420+0000","last_active":"2024-04-26T22:00:44.900537+0000","last_peered":"2024-04-26T22:00:44.900537+0000","last_clean":"2024-04-26T22:00:44.900537+0000","last_became_active":"2024-04-26T21:59:43.059033+0000","last_became_peered":"2024-04-26T21:59:43.059033+0000","last_unstale":"2024-04-26T22:00:44.900537+0000","last_undegraded":"2024-04-26T22:00:44.900537+0000","last_fullsized":"2024-04-26T22:00:44.900537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:57.633543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645231+0000","last_change":"2024-04-26T21:59:44.045932+0000","last_active":"2024-04-26T22:00:44.645231+0000","last_peered":"2024-04-26T22:00:44.645231+0000","last_clean":"2024-04-26T22:00:44.645231+0000","last_became_active":"2024-04-26T21:59:44.044775+0000","last_became_peered":"2024-04-26T21:59:44.044775+0000","last_unstale":"2024-04-26T22:00:44.645231+0000","last_undegraded":"2024-04-26T22:00:44.645231+0000","last_fullsized":"2024-04-26T22:00:44.645231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:05.719002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.655070+0000","last_change":"2024-04-26T21:59:48.151182+0000","last_active":"2024-04-26T22:00:44.655070+0000","last_peered":"2024-04-26T22:00:44.655070+0000","last_clean":"2024-04-26T22:00:44.655070+0000","last_became_active":"2024-04-26T21:59:48.150815+0000","last_became_peered":"2024-04-26T21:59:48.150815+0000","last_unstale":"2024-04-26T22:00:44.655070+0000","last_undegraded":"2024-04-26T22:00:44.655070+0000","last_fullsized":"2024-04-26T22:00:44.655070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.630163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647186+0000","last_change":"2024-04-26T21:59:43.039852+0000","last_active":"2024-04-26T22:00:44.647186+0000","last_peered":"2024-04-26T22:00:44.647186+0000","last_clean":"2024-04-26T22:00:44.647186+0000","last_became_active":"2024-04-26T21:59:43.039499+0000","last_became_peered":"2024-04-26T21:59:43.039499+0000","last_unstale":"2024-04-26T22:00:44.647186+0000","last_undegraded":"2024-04-26T22:00:44.647186+0000","last_fullsized":"2024-04-26T22:00:44.647186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:14.599270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.644898+0000","last_change":"2024-04-26T21:59:44.057285+0000","last_active":"2024-04-26T22:00:44.644898+0000","last_peered":"2024-04-26T22:00:44.644898+0000","last_clean":"2024-04-26T22:00:44.644898+0000","last_became_active":"2024-04-26T21:59:44.056947+0000","last_became_peered":"2024-04-26T21:59:44.056947+0000","last_unstale":"2024-04-26T22:00:44.644898+0000","last_undegraded":"2024-04-26T22:00:44.644898+0000","last_fullsized":"2024-04-26T22:00:44.644898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:58.242075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646714+0000","last_change":"2024-04-26T21:59:48.131624+0000","last_active":"2024-04-26T22:00:44.646714+0000","last_peered":"2024-04-26T22:00:44.646714+0000","last_clean":"2024-04-26T22:00:44.646714+0000","last_became_active":"2024-04-26T21:59:48.131193+0000","last_became_peered":"2024-04-26T21:59:48.131193+0000","last_unstale":"2024-04-26T22:00:44.646714+0000","last_undegraded":"2024-04-26T22:00:44.646714+0000","last_fullsized":"2024-04-26T22:00:44.646714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:47.095790+0000","last_clean_scrub_stamp":"2024-04-26T21:59:47.095790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:52.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.643880+0000","last_change":"2024-04-26T21:59:43.047723+0000","last_active":"2024-04-26T22:00:44.643880+0000","last_peered":"2024-04-26T22:00:44.643880+0000","last_clean":"2024-04-26T22:00:44.643880+0000","last_became_active":"2024-04-26T21:59:43.045876+0000","last_became_peered":"2024-04-26T21:59:43.045876+0000","last_unstale":"2024-04-26T22:00:44.643880+0000","last_undegraded":"2024-04-26T22:00:44.643880+0000","last_fullsized":"2024-04-26T22:00:44.643880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:41.984781+0000","last_clean_scrub_stamp":"2024-04-26T21:59:41.984781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:37.705490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.645171+0000","last_change":"2024-04-26T21:59:44.040723+0000","last_active":"2024-04-26T22:00:44.645171+0000","last_peered":"2024-04-26T22:00:44.645171+0000","last_clean":"2024-04-26T22:00:44.645171+0000","last_became_active":"2024-04-26T21:59:44.040389+0000","last_became_peered":"2024-04-26T21:59:44.040389+0000","last_unstale":"2024-04-26T22:00:44.645171+0000","last_undegraded":"2024-04-26T22:00:44.645171+0000","last_fullsized":"2024-04-26T22:00:44.645171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:05.120695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647186+0000","last_change":"2024-04-26T21:59:44.041027+0000","last_active":"2024-04-26T22:00:44.647186+0000","last_peered":"2024-04-26T22:00:44.647186+0000","last_clean":"2024-04-26T22:00:44.647186+0000","last_became_active":"2024-04-26T21:59:44.040705+0000","last_became_peered":"2024-04-26T21:59:44.040705+0000","last_unstale":"2024-04-26T22:00:44.647186+0000","last_undegraded":"2024-04-26T22:00:44.647186+0000","last_fullsized":"2024-04-26T22:00:44.647186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:11.114850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900928+0000","last_change":"2024-04-26T21:59:44.040916+0000","last_active":"2024-04-26T22:00:44.900928+0000","last_peered":"2024-04-26T22:00:44.900928+0000","last_clean":"2024-04-26T22:00:44.900928+0000","last_became_active":"2024-04-26T21:59:44.040568+0000","last_became_peered":"2024-04-26T21:59:44.040568+0000","last_unstale":"2024-04-26T22:00:44.900928+0000","last_undegraded":"2024-04-26T22:00:44.900928+0000","last_fullsized":"2024-04-26T22:00:44.900928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:04.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900357+0000","last_change":"2024-04-26T21:59:44.046676+0000","last_active":"2024-04-26T22:00:44.900357+0000","last_peered":"2024-04-26T22:00:44.900357+0000","last_clean":"2024-04-26T22:00:44.900357+0000","last_became_active":"2024-04-26T21:59:44.046339+0000","last_became_peered":"2024-04-26T21:59:44.046339+0000","last_unstale":"2024-04-26T22:00:44.900357+0000","last_undegraded":"2024-04-26T22:00:44.900357+0000","last_fullsized":"2024-04-26T22:00:44.900357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:51.964558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.655771+0000","last_change":"2024-04-26T21:59:44.064635+0000","last_active":"2024-04-26T22:00:44.655771+0000","last_peered":"2024-04-26T22:00:44.655771+0000","last_clean":"2024-04-26T22:00:44.655771+0000","last_became_active":"2024-04-26T21:59:44.064174+0000","last_became_peered":"2024-04-26T21:59:44.064174+0000","last_unstale":"2024-04-26T22:00:44.655771+0000","last_undegraded":"2024-04-26T22:00:44.655771+0000","last_fullsized":"2024-04-26T22:00:44.655771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:59.550421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.666626+0000","last_change":"2024-04-26T22:00:44.666626+0000","last_active":"2024-04-26T22:00:44.666626+0000","last_peered":"2024-04-26T22:00:44.666626+0000","last_clean":"2024-04-26T22:00:44.666626+0000","last_became_active":"2024-04-26T22:00:44.666239+0000","last_became_peered":"2024-04-26T22:00:44.666239+0000","last_unstale":"2024-04-26T22:00:44.666626+0000","last_undegraded":"2024-04-26T22:00:44.666626+0000","last_fullsized":"2024-04-26T22:00:44.666626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:13.684014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.648356+0000","last_change":"2024-04-26T21:59:44.044024+0000","last_active":"2024-04-26T22:00:44.648356+0000","last_peered":"2024-04-26T22:00:44.648356+0000","last_clean":"2024-04-26T22:00:44.648356+0000","last_became_active":"2024-04-26T21:59:44.043718+0000","last_became_peered":"2024-04-26T21:59:44.043718+0000","last_unstale":"2024-04-26T22:00:44.648356+0000","last_undegraded":"2024-04-26T22:00:44.648356+0000","last_fullsized":"2024-04-26T22:00:44.648356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:35.028538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.901003+0000","last_change":"2024-04-26T21:59:44.037432+0000","last_active":"2024-04-26T22:00:44.901003+0000","last_peered":"2024-04-26T22:00:44.901003+0000","last_clean":"2024-04-26T22:00:44.901003+0000","last_became_active":"2024-04-26T21:59:44.037086+0000","last_became_peered":"2024-04-26T21:59:44.037086+0000","last_unstale":"2024-04-26T22:00:44.901003+0000","last_undegraded":"2024-04-26T22:00:44.901003+0000","last_fullsized":"2024-04-26T22:00:44.901003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:32.828866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.658899+0000","last_change":"2024-04-26T21:59:44.055253+0000","last_active":"2024-04-26T22:00:44.658899+0000","last_peered":"2024-04-26T22:00:44.658899+0000","last_clean":"2024-04-26T22:00:44.658899+0000","last_became_active":"2024-04-26T21:59:44.054329+0000","last_became_peered":"2024-04-26T21:59:44.054329+0000","last_unstale":"2024-04-26T22:00:44.658899+0000","last_undegraded":"2024-04-26T22:00:44.658899+0000","last_fullsized":"2024-04-26T22:00:44.658899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:02.949272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646654+0000","last_change":"2024-04-26T21:59:44.060213+0000","last_active":"2024-04-26T22:00:44.646654+0000","last_peered":"2024-04-26T22:00:44.646654+0000","last_clean":"2024-04-26T22:00:44.646654+0000","last_became_active":"2024-04-26T21:59:44.059987+0000","last_became_peered":"2024-04-26T21:59:44.059987+0000","last_unstale":"2024-04-26T22:00:44.646654+0000","last_undegraded":"2024-04-26T22:00:44.646654+0000","last_fullsized":"2024-04-26T22:00:44.646654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:23.184373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.651222+0000","last_change":"2024-04-26T21:59:44.046779+0000","last_active":"2024-04-26T22:00:44.651222+0000","last_peered":"2024-04-26T22:00:44.651222+0000","last_clean":"2024-04-26T22:00:44.651222+0000","last_became_active":"2024-04-26T21:59:44.046304+0000","last_became_peered":"2024-04-26T21:59:44.046304+0000","last_unstale":"2024-04-26T22:00:44.651222+0000","last_undegraded":"2024-04-26T22:00:44.651222+0000","last_fullsized":"2024-04-26T22:00:44.651222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:59.738908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.900530+0000","last_change":"2024-04-26T21:59:44.055284+0000","last_active":"2024-04-26T22:00:44.900530+0000","last_peered":"2024-04-26T22:00:44.900530+0000","last_clean":"2024-04-26T22:00:44.900530+0000","last_became_active":"2024-04-26T21:59:44.054897+0000","last_became_peered":"2024-04-26T21:59:44.054897+0000","last_unstale":"2024-04-26T22:00:44.900530+0000","last_undegraded":"2024-04-26T22:00:44.900530+0000","last_fullsized":"2024-04-26T22:00:44.900530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:57.942834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.646627+0000","last_change":"2024-04-26T21:59:44.060516+0000","last_active":"2024-04-26T22:00:44.646627+0000","last_peered":"2024-04-26T22:00:44.646627+0000","last_clean":"2024-04-26T22:00:44.646627+0000","last_became_active":"2024-04-26T21:59:44.059564+0000","last_became_peered":"2024-04-26T21:59:44.059564+0000","last_unstale":"2024-04-26T22:00:44.646627+0000","last_undegraded":"2024-04-26T22:00:44.646627+0000","last_fullsized":"2024-04-26T22:00:44.646627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:43.784018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T22:00:44.647813+0000","last_change":"2024-04-26T21:59:44.043564+0000","last_active":"2024-04-26T22:00:44.647813+0000","last_peered":"2024-04-26T22:00:44.647813+0000","last_clean":"2024-04-26T22:00:44.647813+0000","last_became_active":"2024-04-26T21:59:44.043254+0000","last_became_peered":"2024-04-26T21:59:44.043254+0000","last_unstale":"2024-04-26T22:00:44.647813+0000","last_undegraded":"2024-04-26T22:00:44.647813+0000","last_fullsized":"2024-04-26T22:00:44.647813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:42.999877+0000","last_clean_scrub_stamp":"2024-04-26T21:59:42.999877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:12.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56474,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":185,"num_read_kb":150,"num_write":81,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":169422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":504735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1708032,"data_stored":1155069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":501221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1634304,"data_stored":1099781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":556897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":500729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":556349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":500743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1095285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":55710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":55870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:01:41.083 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T22:01:41.083 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T22:01:41.084 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T22:01:41.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T22:01:41.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:40 smithi154 ceph-mon[32578]: pgmap v380: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:41.202 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:40 smithi012 ceph-mon[28184]: pgmap v380: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:41.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:40 smithi077 ceph-mon[32280]: pgmap v380: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:41.600 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:42.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:41 smithi012 ceph-mon[28184]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:42.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:41 smithi154 ceph-mon[32578]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:42.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:41 smithi077 ceph-mon[32280]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T22:01:43.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:42 smithi012 ceph-mon[28184]: pgmap v381: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:43.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:42 smithi154 ceph-mon[32578]: pgmap v381: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:43.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:42 smithi077 ceph-mon[32280]: pgmap v381: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:43.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 -- 172.21.15.12:0/567362011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10b600 msgr2=0x7fd48c10b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:43.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 --2- 172.21.15.12:0/567362011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10b600 0x7fd48c10b9e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0099c0 tx=0x7fd47c009cd0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 -- 172.21.15.12:0/567362011 shutdown_connections 2024-04-26T22:01:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 --2- 172.21.15.12:0/567362011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 --2- 172.21.15.12:0/567362011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c10b600 0x7fd48c10b9e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 --2- 172.21.15.12:0/567362011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c105550 0x7fd48c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 -- 172.21.15.12:0/567362011 >> 172.21.15.12:0/567362011 conn(0x7fd48c101000 msgr2=0x7fd48c103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:43.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T22:01:43.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:43 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-26T22:01:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 -- 172.21.15.12:0/567362011 shutdown_connections 2024-04-26T22:01:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.940+0000 7fd48be00700 1 -- 172.21.15.12:0/567362011 wait complete. 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.941+0000 7fd48be00700 1 Processor -- start 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.941+0000 7fd48be00700 1 -- start start 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c10b600 0x7fd48c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c115770 con 0x7fd48c105550 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1155f0 con 0x7fd48c10b600 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48be00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1158f0 con 0x7fd48c105f10 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56758/0 (socket says 172.21.15.12:56758) 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48aa00700 1 -- 172.21.15.12:0/2791745497 learned_addr learned my addr 172.21.15.12:0/2791745497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48a000700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.942+0000 7fd48b400700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c10b600 0x7fd48c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 -- 172.21.15.12:0/2791745497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 msgr2=0x7fd48c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 -- 172.21.15.12:0/2791745497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c10b600 msgr2=0x7fd48c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c10b600 0x7fd48c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 -- 172.21.15.12:0/2791745497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19c0f0 con 0x7fd48c105550 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48a000700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.943+0000 7fd48aa00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006010 tx=0x7fd47c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.944+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c026070 con 0x7fd48c105550 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.944+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19c380 con 0x7fd48c105550 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.944+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c005680 con 0x7fd48c105550 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.944+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00fb20 con 0x7fd48c105550 2024-04-26T22:01:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.944+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a86a0 con 0x7fd48c105550 2024-04-26T22:01:43.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.945+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c10da90 con 0x7fd48c105550 2024-04-26T22:01:43.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.947+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd47c029070 con 0x7fd48c105550 2024-04-26T22:01:43.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.949+0000 7fd483400700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 0x7fd4700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:43.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.949+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9158+0+0 (secure 0 0 0) 0x7fd47c0d1140 con 0x7fd48c105550 2024-04-26T22:01:43.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.949+0000 7fd48a000700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 0x7fd4700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:43.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.950+0000 7fd48a000700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 0x7fd4700a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd48c10abc0 tx=0x7fd478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:43.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:43.953+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd47c05e060 con 0x7fd48c105550 2024-04-26T22:01:44.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T22:01:44.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:43 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-26T22:01:44.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T22:01:44.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:43 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-26T22:01:44.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.246+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd48c106c70 con 0x7fd48c105550 2024-04-26T22:01:44.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.248+0000 7fd483400700 1 -- 172.21.15.12:0/2791745497 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd47c013070 con 0x7fd48c105550 2024-04-26T22:01:44.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:44.248 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T22:01:44.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.251+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 msgr2=0x7fd4700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.251+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 0x7fd4700a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd48c10abc0 tx=0x7fd478009260 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.251+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 msgr2=0x7fd48c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.251+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006010 tx=0x7fd47c005280 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.251+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 shutdown_connections 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd48c10b600 0x7fd48c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd48c105550 0x7fd48c1a17d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd4700a3cd0 0x7fd4700a6190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 --2- 172.21.15.12:0/2791745497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd48c105f10 0x7fd48c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 >> 172.21.15.12:0/2791745497 conn(0x7fd48c101000 msgr2=0x7fd48c102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 shutdown_connections 2024-04-26T22:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:44.252+0000 7fd48be00700 1 -- 172.21.15.12:0/2791745497 wait complete. 2024-04-26T22:01:45.028 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T22:01:45.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:01:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:44 smithi154 ceph-mon[32578]: pgmap v382: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-26T22:01:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:44 smithi154 ceph-mon[32578]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-26T22:01:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:44 smithi154 ceph-mon[32578]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:01:45.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:44 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2791745497' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:01:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:44 smithi077 ceph-mon[32280]: pgmap v382: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-26T22:01:45.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:44 smithi077 ceph-mon[32280]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-26T22:01:45.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:44 smithi077 ceph-mon[32280]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:01:45.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:44 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2791745497' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:01:45.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 -- 172.21.15.12:0/3671270709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc105b80 msgr2=0x7f55cc106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 --2- 172.21.15.12:0/3671270709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc105b80 0x7f55cc106000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009900 tx=0x7f55b802a400 comp rx=0 tx=0).stop 2024-04-26T22:01:45.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 -- 172.21.15.12:0/3671270709 shutdown_connections 2024-04-26T22:01:45.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 --2- 172.21.15.12:0/3671270709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc106540 0x7f55cc10cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 --2- 172.21.15.12:0/3671270709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc105b80 0x7f55cc106000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 --2- 172.21.15.12:0/3671270709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc104990 0x7f55cc104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 -- 172.21.15.12:0/3671270709 >> 172.21.15.12:0/3671270709 conn(0x7f55cc0fff00 msgr2=0x7f55cc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:45.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 -- 172.21.15.12:0/3671270709 shutdown_connections 2024-04-26T22:01:45.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.250+0000 7f55d2000640 1 -- 172.21.15.12:0/3671270709 wait complete. 2024-04-26T22:01:45.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.251+0000 7f55d2000640 1 Processor -- start 2024-04-26T22:01:45.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 -- start start 2024-04-26T22:01:45.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 0x7f55cc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc105b80 0x7f55cc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 0x7f55cc1a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1104a0 con 0x7f55cc106540 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc110320 con 0x7f55cc105b80 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d2000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc110620 con 0x7f55cc104990 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d1600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 0x7f55cc1a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55cbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc105b80 0x7f55cc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 0x7f55cc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d1600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 0x7f55cc1a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56786/0 (socket says 172.21.15.12:56786) 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.252+0000 7f55d1600640 1 -- 172.21.15.12:0/3971659173 learned_addr learned my addr 172.21.15.12:0/3971659173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 -- 172.21.15.12:0/3971659173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc105b80 msgr2=0x7f55cc19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc105b80 0x7f55cc19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 -- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 msgr2=0x7f55cc1a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 0x7f55cc1a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 -- 172.21.15.12:0/3971659173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc1a2760 con 0x7f55cc104990 2024-04-26T22:01:45.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55d0c00640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 0x7f55cc19ab40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e9c0 tx=0x7f55c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.253+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000ccc0 con 0x7f55cc104990 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.254+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc1a2a50 con 0x7f55cc104990 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.254+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c00035f0 con 0x7f55cc104990 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.254+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000ce60 con 0x7f55cc104990 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.254+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a2fc0 con 0x7f55cc104990 2024-04-26T22:01:45.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.255+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc10bd30 con 0x7f55cc104990 2024-04-26T22:01:45.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.257+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f55c0004000 con 0x7f55cc104990 2024-04-26T22:01:45.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.259+0000 7f55c9600640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 0x7f55b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.259+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f55c00d0f50 con 0x7f55cc104990 2024-04-26T22:01:45.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.259+0000 7f55cbe00640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 0x7f55b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.260+0000 7f55cbe00640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 0x7f55b40a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f55b8000c00 tx=0x7f55b80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:45.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.261+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f55c009d3d0 con 0x7f55cc104990 2024-04-26T22:01:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:44 smithi012 ceph-mon[28184]: pgmap v382: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:01:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-26T22:01:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:44 smithi012 ceph-mon[28184]: from='mgr.14152 172.21.15.12:0/3401931944' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-26T22:01:45.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:44 smithi012 ceph-mon[28184]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T22:01:45.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:44 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2791745497' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T22:01:45.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.470+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55cc02dd40 con 0x7f55cc104990 2024-04-26T22:01:45.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.472+0000 7f55c9600640 1 -- 172.21.15.12:0/3971659173 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f55c009ccd0 con 0x7f55cc104990 2024-04-26T22:01:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:45.473 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:01:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 msgr2=0x7f55b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 0x7f55b40a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f55b8000c00 tx=0x7f55b80314a0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 msgr2=0x7f55cc19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 0x7f55cc19ab40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e9c0 tx=0x7f55c000ee80 comp rx=0 tx=0).stop 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 shutdown_connections 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f55b40a3c40 0x7f55b40a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc106540 0x7f55cc1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55cc105b80 0x7f55cc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 --2- 172.21.15.12:0/3971659173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55cc104990 0x7f55cc19ab40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 >> 172.21.15.12:0/3971659173 conn(0x7f55cc0fff00 msgr2=0x7f55cc108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 shutdown_connections 2024-04-26T22:01:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.475+0000 7f55d2000640 1 -- 172.21.15.12:0/3971659173 wait complete. 2024-04-26T22:01:45.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:01:45.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 -- 172.21.15.12:0/4183193820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888102980 msgr2=0x7f2888102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 --2- 172.21.15.12:0/4183193820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888102980 0x7f2888102da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2878009900 tx=0x7f287802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 -- 172.21.15.12:0/4183193820 shutdown_connections 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 --2- 172.21.15.12:0/4183193820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888104530 0x7f288810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 --2- 172.21.15.12:0/4183193820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f2888103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 --2- 172.21.15.12:0/4183193820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888102980 0x7f2888102da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 -- 172.21.15.12:0/4183193820 >> 172.21.15.12:0/4183193820 conn(0x7f28880fdf10 msgr2=0x7f2888100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:45.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 -- 172.21.15.12:0/4183193820 shutdown_connections 2024-04-26T22:01:45.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.718+0000 7f288f400640 1 -- 172.21.15.12:0/4183193820 wait complete. 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.719+0000 7f288f400640 1 Processor -- start 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.719+0000 7f288f400640 1 -- start start 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.719+0000 7f288f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888102980 0x7f288819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888104530 0x7f28881a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288f400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f288810e4d0 con 0x7f2888103b70 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288f400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f288810e350 con 0x7f2888102980 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288f400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f288810e650 con 0x7f2888104530 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56806/0 (socket says 172.21.15.12:56806) 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 -- 172.21.15.12:0/306983957 learned_addr learned my addr 172.21.15.12:0/306983957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:45.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888104530 0x7f28881a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f288ca00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888102980 0x7f288819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 -- 172.21.15.12:0/306983957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888104530 msgr2=0x7f28881a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888104530 0x7f28881a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 -- 172.21.15.12:0/306983957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888102980 msgr2=0x7f288819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888102980 0x7f288819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.720+0000 7f287fe00640 1 -- 172.21.15.12:0/306983957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28881a2a30 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f287fe00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f287000e9f0 tx=0x7f287000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287000ccc0 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28700108e0 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28881a2d20 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870018880 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.721+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28881a3210 con 0x7f2888103b70 2024-04-26T22:01:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.722+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28881a3440 con 0x7f2888103b70 2024-04-26T22:01:45.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.725+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2870010430 con 0x7f2888103b70 2024-04-26T22:01:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.726+0000 7f2886000640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 0x7f286c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.726+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f28700d11c0 con 0x7f2888103b70 2024-04-26T22:01:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.726+0000 7f288ca00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 0x7f286c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.727+0000 7f288ca00640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 0x7f286c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2878005ab0 tx=0x7f2878005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.729+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f287009d640 con 0x7f2888103b70 2024-04-26T22:01:45.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.936+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28881a3610 con 0x7f2888103b70 2024-04-26T22:01:45.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.937+0000 7f2886000640 1 -- 172.21.15.12:0/306983957 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f287009cf40 con 0x7f2888103b70 2024-04-26T22:01:45.937 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 msgr2=0x7f286c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 0x7f286c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2878005ab0 tx=0x7f2878005a20 comp rx=0 tx=0).stop 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 msgr2=0x7f288819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f287000e9f0 tx=0x7f287000edf0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 shutdown_connections 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f286c0a3c40 0x7f286c0a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2888104530 0x7f28881a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2888103b70 0x7f288819b350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 --2- 172.21.15.12:0/306983957 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2888102980 0x7f288819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 >> 172.21.15.12:0/306983957 conn(0x7f28880fdf10 msgr2=0x7f28881093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 shutdown_connections 2024-04-26T22:01:45.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:45.940+0000 7f288f400640 1 -- 172.21.15.12:0/306983957 wait complete. 2024-04-26T22:01:45.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:01:46.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:45 smithi154 ceph-mon[32578]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:01:46.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:45 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3971659173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.180+0000 7f014a000640 1 -- 172.21.15.12:0/2693845535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144102a60 msgr2=0x7f0144102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.180+0000 7f014a000640 1 --2- 172.21.15.12:0/2693845535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144102a60 0x7f0144102ee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0130009900 tx=0x7f013002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 -- 172.21.15.12:0/2693845535 shutdown_connections 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 --2- 172.21.15.12:0/2693845535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144103420 0x7f0144109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 --2- 172.21.15.12:0/2693845535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144102a60 0x7f0144102ee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 --2- 172.21.15.12:0/2693845535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f0144101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 -- 172.21.15.12:0/2693845535 >> 172.21.15.12:0/2693845535 conn(0x7f01440fce20 msgr2=0x7f01440ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:46.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 -- 172.21.15.12:0/2693845535 shutdown_connections 2024-04-26T22:01:46.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.181+0000 7f014a000640 1 -- 172.21.15.12:0/2693845535 wait complete. 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 Processor -- start 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 -- start start 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 0x7f014419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144103420 0x7f01441a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f014410c2f0 con 0x7f0144103420 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f014410c170 con 0x7f0144102a60 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.182+0000 7f014a000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f014410c470 con 0x7f0144101870 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0143400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 0x7f014419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0143400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:45266/0 (socket says 172.21.15.12:45266) 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0143400640 1 -- 172.21.15.12:0/2422553560 learned_addr learned my addr 172.21.15.12:0/2422553560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0143e00640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144103420 0x7f01441a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 -- 172.21.15.12:0/2422553560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 msgr2=0x7f014419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 -- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144103420 msgr2=0x7f01441a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144103420 0x7f01441a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0142a00640 1 -- 172.21.15.12:0/2422553560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01441a29d0 con 0x7f0144102a60 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.183+0000 7f0143400640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:46.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f0142a00640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 0x7f014419b2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f013002a9e0 tx=0x7f0130003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013003f070 con 0x7f0144102a60 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0130031c50 con 0x7f0144102a60 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130035030 con 0x7f0144102a60 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01441a2c60 con 0x7f0144102a60 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.184+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01441a30d0 con 0x7f0144102a60 2024-04-26T22:01:46.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.185+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0144108c60 con 0x7f0144102a60 2024-04-26T22:01:46.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.187+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0130042070 con 0x7f0144102a60 2024-04-26T22:01:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.189+0000 7f013be00640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 0x7f01280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.189+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f01300f0fa0 con 0x7f0144102a60 2024-04-26T22:01:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.189+0000 7f0143400640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 0x7f01280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.190+0000 7f0143400640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 0x7f01280a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.193+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f01300bcdd0 con 0x7f0144102a60 2024-04-26T22:01:46.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:45 smithi077 ceph-mon[32280]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:01:46.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:45 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3971659173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:45 smithi012 ceph-mon[28184]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:01:46.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:45 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3971659173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:46.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.407+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f014402dd30 con 0x7f0144102a60 2024-04-26T22:01:46.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.408+0000 7f013be00640 1 -- 172.21.15.12:0/2422553560 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f01300c10d0 con 0x7f0144102a60 2024-04-26T22:01:46.407 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:01:46.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.410+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 msgr2=0x7f01280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.410+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 0x7f01280a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005f90 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.410+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 msgr2=0x7f014419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.410+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 0x7f014419b2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f013002a9e0 tx=0x7f0130003c10 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 shutdown_connections 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f01280a3cc0 0x7f01280a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0144103420 0x7f01441a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0144102a60 0x7f014419b2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 --2- 172.21.15.12:0/2422553560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0144101870 0x7f014419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 >> 172.21.15.12:0/2422553560 conn(0x7f01440fce20 msgr2=0x7f0144105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 shutdown_connections 2024-04-26T22:01:46.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.411+0000 7f014a000640 1 -- 172.21.15.12:0/2422553560 wait complete. 2024-04-26T22:01:46.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:01:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 -- 172.21.15.12:0/295947652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42101033f0 msgr2=0x7f42101057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 --2- 172.21.15.12:0/295947652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42101033f0 0x7f42101057e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009900 tx=0x7f41f802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 -- 172.21.15.12:0/295947652 shutdown_connections 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 --2- 172.21.15.12:0/295947652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4210108650 0x7f421010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 --2- 172.21.15.12:0/295947652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210105d20 0x7f4210108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 --2- 172.21.15.12:0/295947652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42101033f0 0x7f42101057e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.649+0000 7f4215000640 1 -- 172.21.15.12:0/295947652 >> 172.21.15.12:0/295947652 conn(0x7f42100fcde0 msgr2=0x7f42100ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.650+0000 7f4215000640 1 -- 172.21.15.12:0/295947652 shutdown_connections 2024-04-26T22:01:46.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.650+0000 7f4215000640 1 -- 172.21.15.12:0/295947652 wait complete. 2024-04-26T22:01:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 Processor -- start 2024-04-26T22:01:46.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 -- start start 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42101033f0 0x7f4210198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210108650 0x7f42101a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f421010e510 con 0x7f4210108650 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f421010e390 con 0x7f4210105d20 2024-04-26T22:01:46.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.651+0000 7f4215000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f421010e690 con 0x7f42101033f0 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210108650 0x7f42101a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:42456/0 (socket says 172.21.15.12:42456) 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 -- 172.21.15.12:0/829509261 learned_addr learned my addr 172.21.15.12:0/829509261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420de00640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42101033f0 0x7f4210198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 -- 172.21.15.12:0/829509261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42101033f0 msgr2=0x7f4210198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42101033f0 0x7f4210198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 -- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210108650 msgr2=0x7f42101a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210108650 0x7f42101a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 -- 172.21.15.12:0/829509261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42101a07a0 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.652+0000 7f420d400640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f420000eb90 tx=0x7f420000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.653+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200010630 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.653+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101a0a90 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.653+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f420000ee90 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.653+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200018ac0 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.653+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42101a1030 con 0x7f4210105d20 2024-04-26T22:01:46.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.654+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42100fe8d0 con 0x7f4210105d20 2024-04-26T22:01:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.656+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4200018c60 con 0x7f4210105d20 2024-04-26T22:01:46.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.658+0000 7f4206a00640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 0x7f41f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.658+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f42000d0e70 con 0x7f4210105d20 2024-04-26T22:01:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.658+0000 7f420de00640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 0x7f41f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.659+0000 7f420de00640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 0x7f41f40a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41f8031500 tx=0x7f41f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:46.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.660+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f420009cca0 con 0x7f4210105d20 2024-04-26T22:01:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.869+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f421002dcd0 con 0x7f4210105d20 2024-04-26T22:01:46.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.870+0000 7f4206a00640 1 -- 172.21.15.12:0/829509261 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f42000a0fa0 con 0x7f4210105d20 2024-04-26T22:01:46.870 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.872+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 msgr2=0x7f41f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.872+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 0x7f41f40a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41f8031500 tx=0x7f41f8005a20 comp rx=0 tx=0).stop 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 msgr2=0x7f42101990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f420000eb90 tx=0x7f420000c6a0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 shutdown_connections 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f41f40a3cc0 0x7f41f40a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4210108650 0x7f42101a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4210105d20 0x7f42101990c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 --2- 172.21.15.12:0/829509261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42101033f0 0x7f4210198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 >> 172.21.15.12:0/829509261 conn(0x7f42100fcde0 msgr2=0x7f4210103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 shutdown_connections 2024-04-26T22:01:46.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:46.873+0000 7f4215000640 1 -- 172.21.15.12:0/829509261 wait complete. 2024-04-26T22:01:46.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:01:47.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:46 smithi012 ceph-mon[28184]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:47.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:46 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/306983957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:46 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2422553560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.048 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:46 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/829509261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 -- 172.21.15.12:0/3771433964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3750101740 msgr2=0x7f3750101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 --2- 172.21.15.12:0/3771433964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3750101740 0x7f3750101b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3740009900 tx=0x7f374002a410 comp rx=0 tx=0).stop 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 -- 172.21.15.12:0/3771433964 shutdown_connections 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 --2- 172.21.15.12:0/3771433964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37501032f0 0x7f3750109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 --2- 172.21.15.12:0/3771433964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f3750102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 --2- 172.21.15.12:0/3771433964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3750101740 0x7f3750101b60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 -- 172.21.15.12:0/3771433964 >> 172.21.15.12:0/3771433964 conn(0x7f37500fccd0 msgr2=0x7f37500ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 -- 172.21.15.12:0/3771433964 shutdown_connections 2024-04-26T22:01:47.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.080+0000 7f3754a00640 1 -- 172.21.15.12:0/3771433964 wait complete. 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.081+0000 7f3754a00640 1 Processor -- start 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.081+0000 7f3754a00640 1 -- start start 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3750101740 0x7f375019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f375019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f375010d240 con 0x7f37501032f0 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f375010d0c0 con 0x7f3750102930 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f3754a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f375010d3c0 con 0x7f3750101740 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56874/0 (socket says 172.21.15.12:56874) 2024-04-26T22:01:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 -- 172.21.15.12:0/3040463593 learned_addr learned my addr 172.21.15.12:0/3040463593 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374ea00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f375019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374f400640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3750101740 0x7f375019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 -- 172.21.15.12:0/3040463593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3750101740 msgr2=0x7f375019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3750101740 0x7f375019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 -- 172.21.15.12:0/3040463593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 msgr2=0x7f375019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f375019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374fe00640 1 -- 172.21.15.12:0/3040463593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37501a2980 con 0x7f37501032f0 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.082+0000 7f374ea00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f375019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f374fe00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f374800efd0 tx=0x7f374800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748003b70 con 0x7f37501032f0 2024-04-26T22:01:47.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37501a2c70 con 0x7f37501032f0 2024-04-26T22:01:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3748003d10 con 0x7f37501032f0 2024-04-26T22:01:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480108a0 con 0x7f37501032f0 2024-04-26T22:01:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.083+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a31c0 con 0x7f37501032f0 2024-04-26T22:01:47.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.084+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37501a3450 con 0x7f37501032f0 2024-04-26T22:01:47.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.089+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3748004b40 con 0x7f37501032f0 2024-04-26T22:01:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.090+0000 7f3747e00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 0x7f37340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.090+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f37480d1210 con 0x7f37501032f0 2024-04-26T22:01:47.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.091+0000 7f374f400640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 0x7f37340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.091+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f374809d690 con 0x7f37501032f0 2024-04-26T22:01:47.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.092+0000 7f374f400640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 0x7f37340a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3740000c00 tx=0x7f37400314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:46 smithi154 ceph-mon[32578]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:46 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/306983957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:46 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2422553560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:46 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/829509261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:46 smithi077 ceph-mon[32280]: pgmap v385: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:47.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:46 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/306983957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:46 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2422553560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:46 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/829509261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:47.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.301+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f37501a3620 con 0x7f37501032f0 2024-04-26T22:01:47.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.302+0000 7f3747e00640 1 -- 172.21.15.12:0/3040463593 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f374809cf90 con 0x7f37501032f0 2024-04-26T22:01:47.302 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:01:47.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 msgr2=0x7f37340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 0x7f37340a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3740000c00 tx=0x7f37400314c0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 msgr2=0x7f37501a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f374800efd0 tx=0x7f374800c6a0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 shutdown_connections 2024-04-26T22:01:47.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f37340a3c40 0x7f37340a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37501032f0 0x7f37501a2310 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3750102930 0x7f375019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 --2- 172.21.15.12:0/3040463593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3750101740 0x7f375019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 >> 172.21.15.12:0/3040463593 conn(0x7f37500fccd0 msgr2=0x7f3750069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 shutdown_connections 2024-04-26T22:01:47.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.305+0000 7f3754a00640 1 -- 172.21.15.12:0/3040463593 wait complete. 2024-04-26T22:01:47.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:47.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 -- 172.21.15.12:0/3484953515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 msgr2=0x7f4264075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 --2- 172.21.15.12:0/3484953515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f4264075810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4254009900 tx=0x7f425402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 -- 172.21.15.12:0/3484953515 shutdown_connections 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 --2- 172.21.15.12:0/3484953515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f426410cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 --2- 172.21.15.12:0/3484953515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f4264075810 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 --2- 172.21.15.12:0/3484953515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 0x7f4264074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 -- 172.21.15.12:0/3484953515 >> 172.21.15.12:0/3484953515 conn(0x7f42640fdd00 msgr2=0x7f4264100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 -- 172.21.15.12:0/3484953515 shutdown_connections 2024-04-26T22:01:47.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.516+0000 7f426ac00640 1 -- 172.21.15.12:0/3484953515 wait complete. 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.517+0000 7f426ac00640 1 Processor -- start 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.517+0000 7f426ac00640 1 -- start start 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 0x7f426419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f42641a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f42641104b0 con 0x7f426410a630 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4264110330 con 0x7f4264075390 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426ac00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4264110630 con 0x7f4264076970 2024-04-26T22:01:47.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426a200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f42641a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f4269800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426a200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f42641a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56892/0 (socket says 172.21.15.12:56892) 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f4268e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 0x7f426419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f426a200640 1 -- 172.21.15.12:0/1795165372 learned_addr learned my addr 172.21.15.12:0/1795165372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.518+0000 7f4269800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:42498/0 (socket says 172.21.15.12:42498) 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 -- 172.21.15.12:0/1795165372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 msgr2=0x7f426419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 0x7f426419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 -- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 msgr2=0x7f42641a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f42641a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 -- 172.21.15.12:0/1795165372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42641a27c0 con 0x7f4264075390 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4269800640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4258005fd0 tx=0x7f425800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258009c30 con 0x7f4264075390 2024-04-26T22:01:47.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4258009dd0 con 0x7f4264075390 2024-04-26T22:01:47.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258003e20 con 0x7f4264075390 2024-04-26T22:01:47.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.519+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42641a2ab0 con 0x7f4264075390 2024-04-26T22:01:47.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.520+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a2f80 con 0x7f4264075390 2024-04-26T22:01:47.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.521+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4264109360 con 0x7f4264075390 2024-04-26T22:01:47.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.523+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4258004000 con 0x7f4264075390 2024-04-26T22:01:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.524+0000 7f4262000640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 0x7f424c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.524+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f42580d0490 con 0x7f4264075390 2024-04-26T22:01:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.525+0000 7f4268e00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 0x7f424c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.525+0000 7f4268e00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 0x7f424c0a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4254031500 tx=0x7f4254005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.527+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4258017020 con 0x7f4264075390 2024-04-26T22:01:47.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.772+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f426402dad0 con 0x7f4264075390 2024-04-26T22:01:47.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.773+0000 7f4262000640 1 -- 172.21.15.12:0/1795165372 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7f425809c340 con 0x7f4264075390 2024-04-26T22:01:47.774 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:47.774 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:01:47.774 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.776+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 msgr2=0x7f424c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.776+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 0x7f424c0a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4254031500 tx=0x7f4254005a20 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.776+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 msgr2=0x7f426419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.776+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4258005fd0 tx=0x7f425800dde0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 shutdown_connections 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f424c0a3cc0 0x7f424c0a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f426410a630 0x7f42641a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4264076970 0x7f426419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 --2- 172.21.15.12:0/1795165372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4264075390 0x7f426419aba0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 >> 172.21.15.12:0/1795165372 conn(0x7f42640fdd00 msgr2=0x7f4264106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 shutdown_connections 2024-04-26T22:01:47.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.777+0000 7f426ac00640 1 -- 172.21.15.12:0/1795165372 wait complete. 2024-04-26T22:01:47.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:01:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 -- 172.21.15.12:0/1916319229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198102790 msgr2=0x7fc198102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 --2- 172.21.15.12:0/1916319229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198102790 0x7fc198102bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009ca0 tx=0x7fc18c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 -- 172.21.15.12:0/1916319229 shutdown_connections 2024-04-26T22:01:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 --2- 172.21.15.12:0/1916319229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 0x7fc19810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 --2- 172.21.15.12:0/1916319229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198103980 0x7fc198103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 --2- 172.21.15.12:0/1916319229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198102790 0x7fc198102bb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 -- 172.21.15.12:0/1916319229 >> 172.21.15.12:0/1916319229 conn(0x7fc1980fdd00 msgr2=0x7fc198100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 -- 172.21.15.12:0/1916319229 shutdown_connections 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.985+0000 7fc19e800640 1 -- 172.21.15.12:0/1916319229 wait complete. 2024-04-26T22:01:47.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.987+0000 7fc19e800640 1 Processor -- start 2024-04-26T22:01:47.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.987+0000 7fc19e800640 1 -- start start 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 0x7fc1981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 0x7fc19819ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810e2e0 con 0x7fc198103980 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810e160 con 0x7fc198104340 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810e460 con 0x7fc198102790 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 0x7fc1981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56900/0 (socket says 172.21.15.12:56900) 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 0x7fc1981989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:45340/0 (socket says 172.21.15.12:45340) 2024-04-26T22:01:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19ca00640 1 -- 172.21.15.12:0/3087103642 learned_addr learned my addr 172.21.15.12:0/3087103642 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:47.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.988+0000 7fc19de00640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 0x7fc19819ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 -- 172.21.15.12:0/3087103642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 msgr2=0x7fc1981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 0x7fc1981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 -- 172.21.15.12:0/3087103642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 msgr2=0x7fc19819ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 0x7fc19819ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 -- 172.21.15.12:0/3087103642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981a05c0 con 0x7fc198103980 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19ca00640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d950 tx=0x7fc18400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184017030 con 0x7fc198103980 2024-04-26T22:01:47.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184004020 con 0x7fc198103980 2024-04-26T22:01:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1840047a0 con 0x7fc198103980 2024-04-26T22:01:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981a08b0 con 0x7fc198103980 2024-04-26T22:01:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.989+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a0e50 con 0x7fc198103980 2024-04-26T22:01:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.991+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198109b80 con 0x7fc198103980 2024-04-26T22:01:47.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.992+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc184009a30 con 0x7fc198103980 2024-04-26T22:01:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.994+0000 7fc196000640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 0x7fc1800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.994+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc1840d02c0 con 0x7fc198103980 2024-04-26T22:01:47.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.994+0000 7fc19d400640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 0x7fc1800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.995+0000 7fc19d400640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 0x7fc1800a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000f80 tx=0x7fc18c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:47.999+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc18409c7a0 con 0x7fc198103980 2024-04-26T22:01:48.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:47 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3040463593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:48.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:47 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1795165372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:48.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.222+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19802dcf0 con 0x7fc198103980 2024-04-26T22:01:48.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.227+0000 7fc196000640 1 -- 172.21.15.12:0/3087103642 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7fc18409c0a0 con 0x7fc198103980 2024-04-26T22:01:48.227 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:48.227 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 msgr2=0x7fc1800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 0x7fc1800a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000f80 tx=0x7fc18c005f30 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 msgr2=0x7fc198198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d950 tx=0x7fc18400de10 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 shutdown_connections 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc1800a3cc0 0x7fc1800a6180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc198104340 0x7fc19819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc198103980 0x7fc198198ee0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 --2- 172.21.15.12:0/3087103642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc198102790 0x7fc1981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 >> 172.21.15.12:0/3087103642 conn(0x7fc1980fdd00 msgr2=0x7fc198106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 shutdown_connections 2024-04-26T22:01:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.230+0000 7fc19e800640 1 -- 172.21.15.12:0/3087103642 wait complete. 2024-04-26T22:01:48.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:47 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3040463593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:48.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:47 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1795165372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:48.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:01:48.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:01:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:47 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3040463593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:48.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:47 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1795165372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:48.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 -- 172.21.15.12:0/442630748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1023c0 msgr2=0x7f78ec102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 --2- 172.21.15.12:0/442630748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1023c0 0x7f78ec102840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009c90 tx=0x7f78e402a860 comp rx=0 tx=0).stop 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 -- 172.21.15.12:0/442630748 shutdown_connections 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 --2- 172.21.15.12:0/442630748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1023c0 0x7f78ec102840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 --2- 172.21.15.12:0/442630748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 --2- 172.21.15.12:0/442630748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1010c0 0x7f78ec1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.465+0000 7f78f1800640 1 -- 172.21.15.12:0/442630748 >> 172.21.15.12:0/442630748 conn(0x7f78ec0fcce0 msgr2=0x7f78ec0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.466+0000 7f78f1800640 1 -- 172.21.15.12:0/442630748 shutdown_connections 2024-04-26T22:01:48.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.466+0000 7f78f1800640 1 -- 172.21.15.12:0/442630748 wait complete. 2024-04-26T22:01:48.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.466+0000 7f78f1800640 1 Processor -- start 2024-04-26T22:01:48.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 -- start start 2024-04-26T22:01:48.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 0x7f78ec19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1023c0 0x7f78ec1a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec110550 con 0x7f78ec1010c0 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1103d0 con 0x7f78ec1023c0 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78f1800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1106d0 con 0x7f78ec101a20 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78ebe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 0x7f78ec19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:45356/0 (socket says 172.21.15.12:45356) 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.467+0000 7f78e3e00640 1 -- 172.21.15.12:0/285828724 learned_addr learned my addr 172.21.15.12:0/285828724 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:48.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78f0e00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1023c0 0x7f78ec1a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 -- 172.21.15.12:0/285828724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1023c0 msgr2=0x7f78ec1a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1023c0 0x7f78ec1a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 -- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 msgr2=0x7f78ec19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 0x7f78ec19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 -- 172.21.15.12:0/285828724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec1a2860 con 0x7f78ec101a20 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78ebe00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 0x7f78ec19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e3e00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9f0 tx=0x7f78d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.468+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400cd80 con 0x7f78ec101a20 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.469+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec1a2b50 con 0x7f78ec101a20 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.469+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d40108e0 con 0x7f78ec101a20 2024-04-26T22:01:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.469+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4018980 con 0x7f78ec101a20 2024-04-26T22:01:48.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.469+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a30f0 con 0x7f78ec101a20 2024-04-26T22:01:48.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.470+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec10c040 con 0x7f78ec101a20 2024-04-26T22:01:48.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.472+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f78d4010430 con 0x7f78ec101a20 2024-04-26T22:01:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.474+0000 7f78e9600640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 0x7f78d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.474+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f78d40d11f0 con 0x7f78ec101a20 2024-04-26T22:01:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.475+0000 7f78ebe00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 0x7f78d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.475+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f78d409d020 con 0x7f78ec101a20 2024-04-26T22:01:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.475+0000 7f78ebe00640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 0x7f78d00a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f78ec19c120 tx=0x7f78dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.673+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78ec02dc60 con 0x7f78ec101a20 2024-04-26T22:01:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.674+0000 7f78e9600640 1 -- 172.21.15.12:0/285828724 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f78d40a1320 con 0x7f78ec101a20 2024-04-26T22:01:48.674 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 msgr2=0x7f78d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 0x7f78d00a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f78ec19c120 tx=0x7f78dc005f90 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 msgr2=0x7f78ec19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9f0 tx=0x7f78d400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 shutdown_connections 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78d00a3c40 0x7f78d00a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78ec1023c0 0x7f78ec1a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec101a20 0x7f78ec19b190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 --2- 172.21.15.12:0/285828724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78ec1010c0 0x7f78ec19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 >> 172.21.15.12:0/285828724 conn(0x7f78ec0fcce0 msgr2=0x7f78ec0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 shutdown_connections 2024-04-26T22:01:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.677+0000 7f78f1800640 1 -- 172.21.15.12:0/285828724 wait complete. 2024-04-26T22:01:48.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.908+0000 7f1beee00640 1 -- 172.21.15.12:0/4199030719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8105d30 msgr2=0x7f1be8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.908+0000 7f1beee00640 1 --2- 172.21.15.12:0/4199030719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8105d30 0x7f1be8108120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009900 tx=0x7f1bd402a410 comp rx=0 tx=0).stop 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 -- 172.21.15.12:0/4199030719 shutdown_connections 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 --2- 172.21.15.12:0/4199030719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8108660 0x7f1be810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 --2- 172.21.15.12:0/4199030719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8105d30 0x7f1be8108120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 --2- 172.21.15.12:0/4199030719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8103400 0x7f1be81057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 -- 172.21.15.12:0/4199030719 >> 172.21.15.12:0/4199030719 conn(0x7f1be80fcdf0 msgr2=0x7f1be80ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 -- 172.21.15.12:0/4199030719 shutdown_connections 2024-04-26T22:01:48.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.909+0000 7f1beee00640 1 -- 172.21.15.12:0/4199030719 wait complete. 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 Processor -- start 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 -- start start 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 0x7f1be819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8108660 0x7f1be81a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1be810e4f0 con 0x7f1be8105d30 2024-04-26T22:01:48.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1be810e370 con 0x7f1be8103400 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1beee00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1be810e670 con 0x7f1be8108660 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.910+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44040/0 (socket says 172.21.15.12:44040) 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 -- 172.21.15.12:0/54070397 learned_addr learned my addr 172.21.15.12:0/54070397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bece00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8108660 0x7f1be81a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1be7e00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 0x7f1be819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 -- 172.21.15.12:0/54070397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8108660 msgr2=0x7f1be81a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8108660 0x7f1be81a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 -- 172.21.15.12:0/54070397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 msgr2=0x7f1be819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 0x7f1be819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 -- 172.21.15.12:0/54070397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be81a2ac0 con 0x7f1be8105d30 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1be7e00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 0x7f1be819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.911+0000 7f1bdfe00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4031500 tx=0x7f1bd4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.912+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd403f070 con 0x7f1be8105d30 2024-04-26T22:01:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.912+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd4031c60 con 0x7f1be8105d30 2024-04-26T22:01:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.912+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4035030 con 0x7f1be8105d30 2024-04-26T22:01:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.912+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be81a2d50 con 0x7f1be8105d30 2024-04-26T22:01:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.912+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be81a3190 con 0x7f1be8105d30 2024-04-26T22:01:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.913+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be810a1b0 con 0x7f1be8105d30 2024-04-26T22:01:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.915+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1bd4042070 con 0x7f1be8105d30 2024-04-26T22:01:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.916+0000 7f1be5600640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 0x7f1bcc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.916+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1bd40f0c70 con 0x7f1be8105d30 2024-04-26T22:01:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.916+0000 7f1be7e00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 0x7f1bcc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.917+0000 7f1be7e00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 0x7f1bcc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:48.921+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1bd40bd0f0 con 0x7f1be8105d30 2024-04-26T22:01:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:48 smithi012 ceph-mon[28184]: pgmap v386: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:48 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3087103642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:49.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:48 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/285828724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.123+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1be802dd20 con 0x7f1be8105d30 2024-04-26T22:01:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.124+0000 7f1be5600640 1 -- 172.21.15.12:0/54070397 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f1bd40bc9f0 con 0x7f1be8105d30 2024-04-26T22:01:49.124 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:01:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 msgr2=0x7f1bcc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 0x7f1bcc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd8009500 comp rx=0 tx=0).stop 2024-04-26T22:01:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 msgr2=0x7f1be819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4031500 tx=0x7f1bd4003c10 comp rx=0 tx=0).stop 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 shutdown_connections 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f1bcc0a3c40 0x7f1bcc0a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1be8108660 0x7f1be81a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1be8105d30 0x7f1be819b3e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 --2- 172.21.15.12:0/54070397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be8103400 0x7f1be819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 >> 172.21.15.12:0/54070397 conn(0x7f1be80fcdf0 msgr2=0x7f1be8102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.127+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 shutdown_connections 2024-04-26T22:01:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.128+0000 7f1beee00640 1 -- 172.21.15.12:0/54070397 wait complete. 2024-04-26T22:01:49.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:01:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:48 smithi077 ceph-mon[32280]: pgmap v386: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:49.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:48 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3087103642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:49.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:48 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/285828724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 -- 172.21.15.12:0/3310028849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 msgr2=0x7fcc00104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 --2- 172.21.15.12:0/3310028849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc00104db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009900 tx=0x7fcbf002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 -- 172.21.15.12:0/3310028849 shutdown_connections 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 --2- 172.21.15.12:0/3310028849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 0x7fcc0010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 --2- 172.21.15.12:0/3310028849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc00106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 --2- 172.21.15.12:0/3310028849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc00104db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.365+0000 7fcc05200640 1 -- 172.21.15.12:0/3310028849 >> 172.21.15.12:0/3310028849 conn(0x7fcc000fff00 msgr2=0x7fcc00102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.366+0000 7fcc05200640 1 -- 172.21.15.12:0/3310028849 shutdown_connections 2024-04-26T22:01:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.366+0000 7fcc05200640 1 -- 172.21.15.12:0/3310028849 wait complete. 2024-04-26T22:01:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.367+0000 7fcc05200640 1 Processor -- start 2024-04-26T22:01:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.367+0000 7fcc05200640 1 -- start start 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.367+0000 7fcc05200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc0019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcc05200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcc05200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 0x7fcc001a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcc05200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00110510 con 0x7fcc00105b80 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcc05200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00110390 con 0x7fcc00104990 2024-04-26T22:01:49.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcc05200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00110690 con 0x7fcc00106540 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44064/0 (socket says 172.21.15.12:44064) 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 -- 172.21.15.12:0/3557701844 learned_addr learned my addr 172.21.15.12:0/3557701844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbff400640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc0019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbffe00640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 0x7fcc001a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 -- 172.21.15.12:0/3557701844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 msgr2=0x7fcc001a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 0x7fcc001a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 -- 172.21.15.12:0/3557701844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 msgr2=0x7fcc0019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc0019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.368+0000 7fcbfea00640 1 -- 172.21.15.12:0/3557701844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a2800 con 0x7fcc00105b80 2024-04-26T22:01:49.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcbfea00640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00ea60 tx=0x7fcbec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00cde0 con 0x7fcc00105b80 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbec0035f0 con 0x7fcc00105b80 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec010690 con 0x7fcc00105b80 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a2af0 con 0x7fcc00105b80 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.369+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a3090 con 0x7fcc00105b80 2024-04-26T22:01:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.370+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0010bcd0 con 0x7fcc00105b80 2024-04-26T22:01:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.373+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcbec0037a0 con 0x7fcc00105b80 2024-04-26T22:01:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.375+0000 7fcbf7e00640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 0x7fcbe40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.376+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcbec0d0c80 con 0x7fcc00105b80 2024-04-26T22:01:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.376+0000 7fcbff400640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 0x7fcbe40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.377+0000 7fcbff400640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 0x7fcbe40a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005ab0 tx=0x7fcbf0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.380+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcbec09d100 con 0x7fcc00105b80 2024-04-26T22:01:49.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:48 smithi154 ceph-mon[32578]: pgmap v386: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:49.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:48 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3087103642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:49.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:48 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/285828724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:01:49.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.588+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc0002dca0 con 0x7fcc00105b80 2024-04-26T22:01:49.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.589+0000 7fcbf7e00640 1 -- 172.21.15.12:0/3557701844 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fcbec09ca00 con 0x7fcc00105b80 2024-04-26T22:01:49.589 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.591+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 msgr2=0x7fcbe40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.591+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 0x7fcbe40a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005ab0 tx=0x7fcbf0005a20 comp rx=0 tx=0).stop 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.591+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 msgr2=0x7fcc0019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.591+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00ea60 tx=0x7fcbec00ef20 comp rx=0 tx=0).stop 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 shutdown_connections 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fcbe40a3cc0 0x7fcbe40a6180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc00106540 0x7fcc001a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc00105b80 0x7fcc0019b120 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 --2- 172.21.15.12:0/3557701844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc00104990 0x7fcc0019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 >> 172.21.15.12:0/3557701844 conn(0x7fcc000fff00 msgr2=0x7fcc00108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 shutdown_connections 2024-04-26T22:01:49.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.592+0000 7fcc05200640 1 -- 172.21.15.12:0/3557701844 wait complete. 2024-04-26T22:01:49.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:01:49.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 -- 172.21.15.12:0/2235416790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e4108e70 msgr2=0x7f35e4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 --2- 172.21.15.12:0/2235416790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e4108e70 0x7f35e4109290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009900 tx=0x7f35d402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 -- 172.21.15.12:0/2235416790 shutdown_connections 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 --2- 172.21.15.12:0/2235416790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e410a980 0x7f35e41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 --2- 172.21.15.12:0/2235416790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4109fc0 0x7f35e410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 --2- 172.21.15.12:0/2235416790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e4108e70 0x7f35e4109290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 -- 172.21.15.12:0/2235416790 >> 172.21.15.12:0/2235416790 conn(0x7f35e4077e00 msgr2=0x7f35e407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 -- 172.21.15.12:0/2235416790 shutdown_connections 2024-04-26T22:01:49.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.828+0000 7f35e8e00640 1 -- 172.21.15.12:0/2235416790 wait complete. 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.829+0000 7f35e8e00640 1 Processor -- start 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 -- start start 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4108e70 0x7f35e419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e4109fc0 0x7f35e419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41149f0 con 0x7f35e410a980 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4114870 con 0x7f35e4109fc0 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e8e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4114b70 con 0x7f35e4108e70 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44086/0 (socket says 172.21.15.12:44086) 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e3e00640 1 -- 172.21.15.12:0/3347711575 learned_addr learned my addr 172.21.15.12:0/3347711575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35e3400640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4108e70 0x7f35e419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.830+0000 7f35daa00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e4109fc0 0x7f35e419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 -- 172.21.15.12:0/3347711575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4108e70 msgr2=0x7f35e419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4108e70 0x7f35e419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 -- 172.21.15.12:0/3347711575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e4109fc0 msgr2=0x7f35e419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e4109fc0 0x7f35e419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 -- 172.21.15.12:0/3347711575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e41a6cd0 con 0x7f35e410a980 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e3e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00eb10 tx=0x7f35dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc004b20 con 0x7f35e410a980 2024-04-26T22:01:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc004cc0 con 0x7f35e410a980 2024-04-26T22:01:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc010670 con 0x7f35e410a980 2024-04-26T22:01:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e41a6fc0 con 0x7f35e410a980 2024-04-26T22:01:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.831+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41a7480 con 0x7f35e410a980 2024-04-26T22:01:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.833+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e4110180 con 0x7f35e410a980 2024-04-26T22:01:49.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.835+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f35dc005020 con 0x7f35e410a980 2024-04-26T22:01:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.837+0000 7f35e0c00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 0x7f35c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.837+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f35dc0d0e10 con 0x7f35e410a980 2024-04-26T22:01:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.837+0000 7f35e3400640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 0x7f35c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.838+0000 7f35e3400640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 0x7f35c80a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35d4031500 tx=0x7f35d4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:49.841+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f35dc09d2f0 con 0x7f35e410a980 2024-04-26T22:01:50.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.055+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f35e402dd20 con 0x7f35e410a980 2024-04-26T22:01:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.056+0000 7f35e0c00640 1 -- 172.21.15.12:0/3347711575 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f35dc09cbf0 con 0x7f35e410a980 2024-04-26T22:01:50.056 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.058+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 msgr2=0x7f35c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.058+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 0x7f35c80a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35d4031500 tx=0x7f35d4005a20 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.058+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 msgr2=0x7f35e41a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.058+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00eb10 tx=0x7f35dc00c2d0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 shutdown_connections 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f35c80a3cc0 0x7f35c80a6180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35e410a980 0x7f35e41a6660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e4109fc0 0x7f35e419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 --2- 172.21.15.12:0/3347711575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e4108e70 0x7f35e419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 >> 172.21.15.12:0/3347711575 conn(0x7f35e4077e00 msgr2=0x7f35e410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 shutdown_connections 2024-04-26T22:01:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.059+0000 7f35e8e00640 1 -- 172.21.15.12:0/3347711575 wait complete. 2024-04-26T22:01:50.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:50.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:49 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/54070397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:49 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3557701844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 -- 172.21.15.12:0/3098008891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 msgr2=0x7fbe5010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/3098008891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe5010cdf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009ca0 tx=0x7fbe4802a8b0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 -- 172.21.15.12:0/3098008891 shutdown_connections 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/3098008891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe5010cdf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/3098008891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe50106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/3098008891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe50104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 -- 172.21.15.12:0/3098008891 >> 172.21.15.12:0/3098008891 conn(0x7fbe500fff40 msgr2=0x7fbe50102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 -- 172.21.15.12:0/3098008891 shutdown_connections 2024-04-26T22:01:50.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.284+0000 7fbe4fe00640 1 -- 172.21.15.12:0/3098008891 wait complete. 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.285+0000 7fbe4fe00640 1 Processor -- start 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.285+0000 7fbe4fe00640 1 -- start start 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe5019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe501a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50110530 con 0x7fbe50106560 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501103b0 con 0x7fbe501049b0 2024-04-26T22:01:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501106b0 con 0x7fbe50105ba0 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe501a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe5019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:45550/0 (socket says 172.21.15.12:45550) 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 -- 172.21.15.12:0/338115766 learned_addr learned my addr 172.21.15.12:0/338115766 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4e000640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe5019b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:59628/0 (socket says 172.21.15.12:59628) 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 -- 172.21.15.12:0/338115766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 msgr2=0x7fbe5019b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe5019b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 -- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 msgr2=0x7fbe501a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe501a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.286+0000 7fbe4ea00640 1 -- 172.21.15.12:0/338115766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe501a2730 con 0x7fbe501049b0 2024-04-26T22:01:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe4ea00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9c0 tx=0x7fbe4000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000cd80 con 0x7fbe501049b0 2024-04-26T22:01:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe400035f0 con 0x7fbe501049b0 2024-04-26T22:01:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe501a2a20 con 0x7fbe501049b0 2024-04-26T22:01:50.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40010690 con 0x7fbe501049b0 2024-04-26T22:01:50.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.287+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501a2fc0 con 0x7fbe501049b0 2024-04-26T22:01:50.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.289+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe50069670 con 0x7fbe501049b0 2024-04-26T22:01:50.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.290+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe400037a0 con 0x7fbe501049b0 2024-04-26T22:01:50.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.292+0000 7fbe47400640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 0x7fbe340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.292+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbe400d0c70 con 0x7fbe501049b0 2024-04-26T22:01:50.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.292+0000 7fbe4e000640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 0x7fbe340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.293+0000 7fbe4e000640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 0x7fbe340a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe5019c020 tx=0x7fbe3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.296+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe4009d0f0 con 0x7fbe501049b0 2024-04-26T22:01:50.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:49 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/54070397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:49 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3557701844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:49 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/54070397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:49 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3557701844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:01:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.545+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe5010bc70 con 0x7fbe501049b0 2024-04-26T22:01:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.547+0000 7fbe47400640 1 -- 172.21.15.12:0/338115766 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7fbe4009c9f0 con 0x7fbe501049b0 2024-04-26T22:01:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:01:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 msgr2=0x7fbe340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 0x7fbe340a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbe5019c020 tx=0x7fbe3c005f90 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 msgr2=0x7fbe5019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9c0 tx=0x7fbe4000ee80 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 shutdown_connections 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe340a3cc0 0x7fbe340a6180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe50106560 0x7fbe501a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe50105ba0 0x7fbe5019b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 --2- 172.21.15.12:0/338115766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe501049b0 0x7fbe5019ab10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.549+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 >> 172.21.15.12:0/338115766 conn(0x7fbe500fff40 msgr2=0x7fbe50108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.550+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 shutdown_connections 2024-04-26T22:01:50.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.550+0000 7fbe4fe00640 1 -- 172.21.15.12:0/338115766 wait complete. 2024-04-26T22:01:50.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 -- 172.21.15.12:0/2100977236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 msgr2=0x7f86e0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 --2- 172.21.15.12:0/2100977236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e0105fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a410 comp rx=0 tx=0).stop 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 -- 172.21.15.12:0/2100977236 shutdown_connections 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 --2- 172.21.15.12:0/2100977236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 --2- 172.21.15.12:0/2100977236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e0105fd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 --2- 172.21.15.12:0/2100977236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 0x7f86e0104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 -- 172.21.15.12:0/2100977236 >> 172.21.15.12:0/2100977236 conn(0x7f86e00ffeb0 msgr2=0x7f86e0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 -- 172.21.15.12:0/2100977236 shutdown_connections 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.790+0000 7f86e4a00640 1 -- 172.21.15.12:0/2100977236 wait complete. 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 Processor -- start 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 -- start start 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 0x7f86e019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86e01104e0 con 0x7f86e0105b50 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0110360 con 0x7f86e0106510 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.791+0000 7f86e4a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0110660 con 0x7f86e0104960 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86dde00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86dde00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44118/0 (socket says 172.21.15.12:44118) 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:45572/0 (socket says 172.21.15.12:45572) 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86dde00640 1 -- 172.21.15.12:0/1714934124 learned_addr learned my addr 172.21.15.12:0/1714934124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:50.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86de800640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 0x7f86e019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 -- 172.21.15.12:0/1714934124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 msgr2=0x7f86e019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 0x7f86e019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 -- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 msgr2=0x7f86e019b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e019b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.792+0000 7f86df200640 1 -- 172.21.15.12:0/1714934124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e01a27b0 con 0x7f86e0106510 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86df200640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009fd0 tx=0x7f86d800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d800ce20 con 0x7f86e0106510 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e01a2aa0 con 0x7f86e0106510 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d8005020 con 0x7f86e0106510 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8010690 con 0x7f86e0106510 2024-04-26T22:01:50.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.793+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01a2ff0 con 0x7f86e0106510 2024-04-26T22:01:50.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.794+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e010bca0 con 0x7f86e0106510 2024-04-26T22:01:50.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.796+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86d80054d0 con 0x7f86e0106510 2024-04-26T22:01:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.798+0000 7f86d7400640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.798+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f86d80d0ff0 con 0x7f86e0106510 2024-04-26T22:01:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.798+0000 7f86de800640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.799+0000 7f86de800640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 0x7f86c40a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f86e01059b0 tx=0x7f86d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:50.802+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f86d809d4d0 con 0x7f86e0106510 2024-04-26T22:01:51.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.012+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86e0063530 con 0x7f86e0106510 2024-04-26T22:01:51.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.014+0000 7f86d7400640 1 -- 172.21.15.12:0/1714934124 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f86d809cdd0 con 0x7f86e0106510 2024-04-26T22:01:51.015 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:51.015 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:01:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.017+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 msgr2=0x7f86c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.017+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 0x7f86c40a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f86e01059b0 tx=0x7f86d0005f90 comp rx=0 tx=0).stop 2024-04-26T22:01:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.017+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 msgr2=0x7f86e01a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.017+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009fd0 tx=0x7f86d800ef10 comp rx=0 tx=0).stop 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 shutdown_connections 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f86c40a3c40 0x7f86c40a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86e0106510 0x7f86e01a2140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86e0105b50 0x7f86e019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 --2- 172.21.15.12:0/1714934124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86e0104960 0x7f86e019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 >> 172.21.15.12:0/1714934124 conn(0x7f86e00ffeb0 msgr2=0x7f86e0108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 shutdown_connections 2024-04-26T22:01:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.018+0000 7f86e4a00640 1 -- 172.21.15.12:0/1714934124 wait complete. 2024-04-26T22:01:51.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:50 smithi077 ceph-mon[32280]: pgmap v387: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3347711575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:51.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:50 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/338115766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:51.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 -- 172.21.15.12:0/3145769630 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4106560 msgr2=0x7f70b410cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 --2- 172.21.15.12:0/3145769630 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4106560 0x7f70b410cdf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f709c009900 tx=0x7f709c02a400 comp rx=0 tx=0).stop 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 -- 172.21.15.12:0/3145769630 shutdown_connections 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 --2- 172.21.15.12:0/3145769630 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4106560 0x7f70b410cdf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 --2- 172.21.15.12:0/3145769630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4105ba0 0x7f70b4106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 --2- 172.21.15.12:0/3145769630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 0x7f70b4104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 -- 172.21.15.12:0/3145769630 >> 172.21.15.12:0/3145769630 conn(0x7f70b40fff40 msgr2=0x7f70b4102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.246+0000 7f70ba200640 1 -- 172.21.15.12:0/3145769630 shutdown_connections 2024-04-26T22:01:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.247+0000 7f70ba200640 1 -- 172.21.15.12:0/3145769630 wait complete. 2024-04-26T22:01:51.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.247+0000 7f70ba200640 1 Processor -- start 2024-04-26T22:01:51.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 -- start start 2024-04-26T22:01:51.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 0x7f70b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4105ba0 0x7f70b419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41104c0 con 0x7f70b4106560 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110340 con 0x7f70b41049b0 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70ba200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110640 con 0x7f70b4105ba0 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70b9800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70b9800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44142/0 (socket says 172.21.15.12:44142) 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70b9800640 1 -- 172.21.15.12:0/1204060790 learned_addr learned my addr 172.21.15.12:0/1204060790 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70b3e00640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4105ba0 0x7f70b419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.248+0000 7f70b8e00640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 0x7f70b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 -- 172.21.15.12:0/1204060790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4105ba0 msgr2=0x7f70b419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4105ba0 0x7f70b419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 -- 172.21.15.12:0/1204060790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 msgr2=0x7f70b419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 0x7f70b419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 -- 172.21.15.12:0/1204060790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b41a6b10 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b9800640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f709c031500 tx=0x7f709c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c03f030 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b41a6da0 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.249+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c003cb0 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.250+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c035070 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.250+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41a7230 con 0x7f70b4106560 2024-04-26T22:01:51.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.251+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b41a7460 con 0x7f70b4106560 2024-04-26T22:01:51.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.253+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f709c042030 con 0x7f70b4106560 2024-04-26T22:01:51.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.254+0000 7f70b1600640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 0x7f70980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.254+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f709c0f0d90 con 0x7f70b4106560 2024-04-26T22:01:51.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.254+0000 7f70b8e00640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 0x7f70980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.255+0000 7f70b8e00640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 0x7f70980a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70b4105a00 tx=0x7f70ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.258+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f709c0bcbc0 con 0x7f70b4106560 2024-04-26T22:01:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:50 smithi012 ceph-mon[28184]: pgmap v387: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3347711575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:51.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:50 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/338115766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:50 smithi154 ceph-mon[32578]: pgmap v387: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3347711575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:01:51.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:50 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/338115766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:51.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.507+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70b41a7800 con 0x7f70b4106560 2024-04-26T22:01:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.509+0000 7f70b1600640 1 -- 172.21.15.12:0/1204060790 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7f709c0c0ec0 con 0x7f70b4106560 2024-04-26T22:01:51.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:01:51.509 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:01:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:01:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.511+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 msgr2=0x7f70980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 0x7f70980a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70b4105a00 tx=0x7f70ac009500 comp rx=0 tx=0).stop 2024-04-26T22:01:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 msgr2=0x7f70b41a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f709c031500 tx=0x7f709c0042f0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 shutdown_connections 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f70980a3c40 0x7f70980a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70b4106560 0x7f70b41a64a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70b4105ba0 0x7f70b419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 --2- 172.21.15.12:0/1204060790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b41049b0 0x7f70b419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 >> 172.21.15.12:0/1204060790 conn(0x7f70b40fff40 msgr2=0x7f70b4078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 shutdown_connections 2024-04-26T22:01:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.512+0000 7f70ba200640 1 -- 172.21.15.12:0/1204060790 wait complete. 2024-04-26T22:01:51.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:59:43.592535+0000', 'modified': '2024-04-26T22:00:09.736381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24565': {'gid': 24565, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/3734088968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 3734088968}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3734088968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6839/4217479283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4217479283}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4217479283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-26T22:01:51.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:01:51.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 -- 172.21.15.12:0/1441036412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 msgr2=0x7fdfcc10cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/1441036412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc10cdb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc402a410 comp rx=0 tx=0).stop 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 -- 172.21.15.12:0/1441036412 shutdown_connections 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/1441036412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc10cdb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/1441036412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 0x7fdfcc105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/1441036412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 -- 172.21.15.12:0/1441036412 >> 172.21.15.12:0/1441036412 conn(0x7fdfcc0ffec0 msgr2=0x7fdfcc102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 -- 172.21.15.12:0/1441036412 shutdown_connections 2024-04-26T22:01:51.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.751+0000 7fdfcbe00640 1 -- 172.21.15.12:0/1441036412 wait complete. 2024-04-26T22:01:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.752+0000 7fdfcbe00640 1 Processor -- start 2024-04-26T22:01:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.752+0000 7fdfcbe00640 1 -- start start 2024-04-26T22:01:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 0x7fdfcc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc1a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1104a0 con 0x7fdfcc106520 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc110320 con 0x7fdfcc104970 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcbe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc110620 con 0x7fdfcc105b60 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:45608/0 (socket says 172.21.15.12:45608) 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 -- 172.21.15.12:0/3400726656 learned_addr learned my addr 172.21.15.12:0/3400726656 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfca000640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 0x7fdfcc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcb400640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc1a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 -- 172.21.15.12:0/3400726656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 msgr2=0x7fdfcc19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 0x7fdfcc19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 -- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 msgr2=0x7fdfcc1a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc1a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.753+0000 7fdfcaa00640 1 -- 172.21.15.12:0/3400726656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a2760 con 0x7fdfcc104970 2024-04-26T22:01:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfcaa00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9c0 tx=0x7fdfbc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfd0a00640 1 -- 172.21.15.12:0/3400726656 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00cde0 con 0x7fdfcc104970 2024-04-26T22:01:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfd0a00640 1 -- 172.21.15.12:0/3400726656 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc0108e0 con 0x7fdfcc104970 2024-04-26T22:01:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a2a50 con 0x7fdfcc104970 2024-04-26T22:01:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfd0a00640 1 -- 172.21.15.12:0/3400726656 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc018960 con 0x7fdfcc104970 2024-04-26T22:01:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.754+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a2fa0 con 0x7fdfcc104970 2024-04-26T22:01:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.757+0000 7fdfd4d91740 1 Processor -- start 2024-04-26T22:01:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 -- start start 2024-04-26T22:01:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd0a00640 1 -- 172.21.15.12:0/3400726656 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdfbc01f020 con 0x7fdfcc104970 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b9b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564db9af7600 con 0x564db9b93cb0 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564db9a94100 con 0x564db9b932c0 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564db9af35c0 con 0x564db9b98e20 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfcaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:45610/0 (socket says 172.21.15.12:45610) 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.758+0000 7fdfcb400640 1 -- 172.21.15.12:0/1086861965 learned_addr learned my addr 172.21.15.12:0/1086861965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfca000640 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b9b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 -- 172.21.15.12:0/1086861965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 msgr2=0x564db9b9b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b9b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 -- 172.21.15.12:0/1086861965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 msgr2=0x564db9b987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 -- 172.21.15.12:0/1086861965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db98092d0 con 0x564db9b932c0 2024-04-26T22:01:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b716da0ba196c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfb7e00640 1 -- 172.21.15.12:0/1086861965 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4035030 con 0x564db9b932c0 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfb7e00640 1 -- 172.21.15.12:0/1086861965 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc4040070 con 0x564db9b932c0 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfb7e00640 1 -- 172.21.15.12:0/1086861965 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003910 con 0x564db9b932c0 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.759+0000 7fdfd0a00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 0x7fdfb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd0a00640 1 -- 172.21.15.12:0/3400726656 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfbc0d0c00 con 0x7fdfcc104970 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1086861965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 msgr2=0x564db9b936e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc4005dd0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfcaa00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 0x7fdfb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1086861965 shutdown_connections 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b9b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1086861965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b936e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1086861965 >> 172.21.15.12:0/1086861965 conn(0x564db9b27d10 msgr2=0x564db9b8ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1086861965 shutdown_connections 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.760+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1086861965 wait complete. 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.761+0000 7fdfcaa00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 0x7fdfb00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564db9b940b0 tx=0x7fdfbc0d46b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.761+0000 7fdfd4d91740 1 -- start start 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.761+0000 7fdfd4d91740 1 Processor -- start 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.761+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b932c0 0x564db9b9d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.761+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 0x564db9ba4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564db9b9bd70 con 0x564db9b93cb0 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564db9a7ef70 con 0x564db9b98e20 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564db9b9bef0 con 0x564db9b932c0 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfd4d91740 10 client.0 _ll_register_callbacks cb 0x564db9aed480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b932c0 0x564db9b9d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44172/0 (socket says 172.21.15.12:44172) 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 -- 172.21.15.12:0/1003573475 learned_addr learned my addr 172.21.15.12:0/1003573475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 0x564db9ba4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 -- 172.21.15.12:0/1003573475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b932c0 msgr2=0x564db9b9d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b932c0 0x564db9b9d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 -- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 msgr2=0x564db9ba4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 0x564db9ba4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfca000640 1 -- 172.21.15.12:0/1003573475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db98d36b0 con 0x564db9b93cb0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.762+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 0x564db9ba4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfca000640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8012960 tx=0x7fdfb8012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7816bd69d981c211 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800f980 con 0x564db9b93cb0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc4040070 con 0x564db9b93cb0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfb800fb20 con 0x564db9b93cb0 2024-04-26T22:01:51.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdfb8025050 con 0x564db9b93cb0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfd4d91740 10 client.15042 fetch_fsmap learned FSMap version 21 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.763+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564db9ba7d20 con 0x564db9b93cb0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc4040070 con 0x564db9b93cb0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdfb8004450 con 0x564db9b93cb0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 10 client.15042 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 20 client.15042 populate_metadata read hostname 'smithi012' 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9baa3b0 0x564db9bab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 4 client.15042 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.764+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db9b05210 con 0x564db9baa3b0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.765+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9baa3b0 0x564db9bab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.765+0000 7fdfcb400640 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9baa3b0 0x564db9bab820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29628a05631d63d7 server_cookie=1d88e51796cde0e3 in_seq=0 out_seq=0 2024-04-26T22:01:51.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.765+0000 7fdfb4c00640 10 client.15042 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfb4c00640 1 -- 172.21.15.12:0/1003573475 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564db9b05210 con 0x564db9baa3b0 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfb4c00640 10 client.15042 handle_command_reply: tid=0 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 client.15042 shutdown 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 msgr2=0x564db9b9dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8012960 tx=0x7fdfb8012e20 comp rx=0 tx=0).stop 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 shutdown_connections 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9baa3b0 0x564db9bab820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b98e20 0x564db9ba4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9b9dc30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/1003573475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b932c0 0x564db9b9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 >> 172.21.15.12:0/1003573475 conn(0x564db9b27d10 msgr2=0x564db9b8df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfb4c00640 0 client.15042 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:01:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 shutdown_connections 2024-04-26T22:01:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.766+0000 7fdfd4d91740 1 -- 172.21.15.12:0/1003573475 wait complete. 2024-04-26T22:01:51.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.767+0000 7fdfd4d91740 20 client.15042 trim_cache size 0 max 16384 2024-04-26T22:01:51.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 Processor -- start 2024-04-26T22:01:51.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 -- start start 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b91d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b92250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564db9ba9b60 con 0x564db9b93cb0 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564db9baa230 con 0x564db9b932c0 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.787+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564db9baa900 con 0x564db9b98e20 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b91d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44174/0 (socket says 172.21.15.12:44174) 2024-04-26T22:01:51.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 -- 172.21.15.12:0/885951856 learned_addr learned my addr 172.21.15.12:0/885951856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfcb400640 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b92250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 -- 172.21.15.12:0/885951856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 msgr2=0x564db9b92250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b92250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 -- 172.21.15.12:0/885951856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 msgr2=0x564db9b91d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b91d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 -- 172.21.15.12:0/885951856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db9b94540 con 0x564db9b93cb0 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfcb400640 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b92250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:51.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.788+0000 7fdfca000640 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e8c984886a6e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfb7400640 1 -- 172.21.15.12:0/885951856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb802d030 con 0x564db9b93cb0 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfb7400640 1 -- 172.21.15.12:0/885951856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb800f980 con 0x564db9b93cb0 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfb7400640 1 -- 172.21.15.12:0/885951856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80051a0 con 0x564db9b93cb0 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 -- 172.21.15.12:0/885951856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 msgr2=0x564db9ba4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb8009240 comp rx=0 tx=0).stop 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 -- 172.21.15.12:0/885951856 shutdown_connections 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b98e20 0x564db9b92250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b93cb0 0x564db9ba4c30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/885951856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9b91d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 -- 172.21.15.12:0/885951856 >> 172.21.15.12:0/885951856 conn(0x564db9b27d10 msgr2=0x564db9b950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.789+0000 7fdfd4d91740 1 -- 172.21.15.12:0/885951856 shutdown_connections 2024-04-26T22:01:51.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.790+0000 7fdfd4d91740 1 -- 172.21.15.12:0/885951856 wait complete. 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.790+0000 7fdfd4d91740 1 -- start start 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.790+0000 7fdfd4d91740 1 Processor -- start 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9ba07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 0x564db9ba0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b98e20 0x564db9ba3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564db9bab6a0 con 0x564db9b98e20 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564db9baafd0 con 0x564db9b932c0 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564db9babd70 con 0x564db9b93cb0 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfd4d91740 10 client.0 _ll_register_callbacks cb 0x7fdfb8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9ba07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9ba07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:45640/0 (socket says 172.21.15.12:45640) 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfcb400640 1 -- 172.21.15.12:0/2294663060 learned_addr learned my addr 172.21.15.12:0/2294663060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 0x564db9ba0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfcb400640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b98e20 0x564db9ba3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 -- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 msgr2=0x564db9ba07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9ba07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 -- 172.21.15.12:0/2294663060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b98e20 msgr2=0x564db9ba3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b98e20 0x564db9ba3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.791+0000 7fdfca000640 1 -- 172.21.15.12:0/2294663060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db9ba4340 con 0x564db9b93cb0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfca000640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 0x564db9ba0d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8004960 tx=0x7fdfb80092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be09b539e6950f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8037060 con 0x564db9b93cb0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfb8004cb0 con 0x564db9b93cb0 2024-04-26T22:01:51.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb800f980 con 0x564db9b93cb0 2024-04-26T22:01:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdfb803e050 con 0x564db9b93cb0 2024-04-26T22:01:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.792+0000 7fdfd4d91740 10 client.25159 fetch_fsmap learned FSMap version 21 2024-04-26T22:01:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564db9c10730 con 0x564db9b93cb0 2024-04-26T22:01:51.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb800f980 con 0x564db9b93cb0 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdfb8032620 con 0x564db9b93cb0 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfd4d91740 10 client.25159 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfd4d91740 10 client.25159 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfd4d91740 20 client.25159 populate_metadata read hostname 'smithi012' 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.793+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9c15000 0x564db9c13d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.794+0000 7fdfd4d91740 4 client.25159 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.794+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "status"}) -- 0x564db9aff080 con 0x564db9c15000 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.794+0000 7fdfcb400640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9c15000 0x564db9c13d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.794+0000 7fdfcb400640 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9c15000 0x564db9c13d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b69a6483ca1eab3 server_cookie=8dcb352bf8f53fb5 in_seq=0 out_seq=0 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.794+0000 7fdfb4c00640 10 client.25159 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfb4c00640 1 -- 172.21.15.12:0/2294663060 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564db9aff080 con 0x564db9c15000 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfb4c00640 10 client.25159 handle_command_reply: tid=0 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 client.25159 shutdown 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 msgr2=0x564db9ba0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 0x564db9ba0d30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8004960 tx=0x7fdfb80092b0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 shutdown_connections 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x564db9c15000 0x564db9c13d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564db9b98e20 0x564db9ba3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564db9b93cb0 0x564db9ba0d30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 --2- 172.21.15.12:0/2294663060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564db9b932c0 0x564db9ba07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 >> 172.21.15.12:0/2294663060 conn(0x564db9b27d10 msgr2=0x564db9b8d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfb4c00640 0 client.25159 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 shutdown_connections 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 1 -- 172.21.15.12:0/2294663060 wait complete. 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.795+0000 7fdfd4d91740 20 client.25159 trim_cache size 0 max 16384 2024-04-26T22:01:51.798 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24581, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 127.725034196, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 21, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 94, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 134.19808820399999 2024-04-26T22:01:51.799 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:01:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 msgr2=0x7fdfb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 0x7fdfb00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564db9b940b0 tx=0x7fdfbc0d46b0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 msgr2=0x7fdfcc19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9c0 tx=0x7fdfbc00ee80 comp rx=0 tx=0).stop 2024-04-26T22:01:51.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 shutdown_connections 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdfb00a3c40 0x7fdfb00a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfcc106520 0x7fdfcc1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc105b60 0x7fdfcc19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 --2- 172.21.15.12:0/3400726656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfcc104970 0x7fdfcc19ab40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 >> 172.21.15.12:0/3400726656 conn(0x7fdfcc0ffec0 msgr2=0x7fdfcc108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 shutdown_connections 2024-04-26T22:01:51.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:51.796+0000 7fdfcbe00640 1 -- 172.21.15.12:0/3400726656 wait complete. 2024-04-26T22:01:51.815 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T22:01:51.816 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T22:01:51.826 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:01:51.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T22:01:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:51 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1714934124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:52.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:51 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1204060790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:51 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1714934124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:52.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:51 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1204060790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:52.303 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:52.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:51 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1714934124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:01:52.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:51 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1204060790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:53.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:52 smithi012 ceph-mon[28184]: pgmap v388: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:53.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:52 smithi077 ceph-mon[32280]: pgmap v388: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:53.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:52 smithi154 ceph-mon[32578]: pgmap v388: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 -- 172.21.15.12:0/323940171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1030a0 msgr2=0x7ff26c103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 --2- 172.21.15.12:0/323940171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1030a0 0x7ff26c103520 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b0a0 tx=0x7ff26400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 -- 172.21.15.12:0/323940171 shutdown_connections 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 --2- 172.21.15.12:0/323940171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1030a0 0x7ff26c103520 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 --2- 172.21.15.12:0/323940171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c101d80 0x7ff26c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 --2- 172.21.15.12:0/323940171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c1026e0 0x7ff26c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 -- 172.21.15.12:0/323940171 >> 172.21.15.12:0/323940171 conn(0x7ff26c0fd920 msgr2=0x7ff26c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 -- 172.21.15.12:0/323940171 shutdown_connections 2024-04-26T22:01:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.639+0000 7ff272800700 1 -- 172.21.15.12:0/323940171 wait complete. 2024-04-26T22:01:54.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.640+0000 7ff272800700 1 Processor -- start 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 -- start start 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 0x7ff26c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 0x7ff26c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 0x7ff26c1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111130 con 0x7ff26c1026e0 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c110fb0 con 0x7ff26c101d80 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff272800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1112b0 con 0x7ff26c1030a0 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff26aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 0x7ff26c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff26aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 0x7ff26c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44200/0 (socket says 172.21.15.12:44200) 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff26b400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 0x7ff26c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff26aa00700 1 -- 172.21.15.12:0/3274405408 learned_addr learned my addr 172.21.15.12:0/3274405408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.641+0000 7ff26be00700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 0x7ff26c1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 -- 172.21.15.12:0/3274405408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 msgr2=0x7ff26c1a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 0x7ff26c1a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 -- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 msgr2=0x7ff26c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 0x7ff26c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 -- 172.21.15.12:0/3274405408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c1a3470 con 0x7ff26c101d80 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26be00700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 0x7ff26c1a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff26b400700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 0x7ff26c19b810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e9f0 tx=0x7ff25c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00c9c0 con 0x7ff26c101d80 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25c00cb60 con 0x7ff26c101d80 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c018680 con 0x7ff26c101d80 2024-04-26T22:01:54.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.642+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c1a3760 con 0x7ff26c101d80 2024-04-26T22:01:54.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.643+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a3ca0 con 0x7ff26c101d80 2024-04-26T22:01:54.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.644+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c06a110 con 0x7ff26c101d80 2024-04-26T22:01:54.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.646+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff25c010430 con 0x7ff26c101d80 2024-04-26T22:01:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.647+0000 7ff263e00700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 0x7ff2500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.647+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff25c0d0aa0 con 0x7ff26c101d80 2024-04-26T22:01:54.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.648+0000 7ff26aa00700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 0x7ff2500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.648+0000 7ff26aa00700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 0x7ff2500a6190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff258009ca0 tx=0x7ff258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.652+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff25c09cf20 con 0x7ff26c101d80 2024-04-26T22:01:54.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.906+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff26c063860 con 0x7ff2500a3cd0 2024-04-26T22:01:54.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.908+0000 7ff263e00700 1 -- 172.21.15.12:0/3274405408 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff26c063860 con 0x7ff2500a3cd0 2024-04-26T22:01:54.908 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-04-26T22:01:54.908 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-04-26T22:01:54.908 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-04-26T22:01:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 msgr2=0x7ff2500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 0x7ff2500a6190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff258009ca0 tx=0x7ff258009260 comp rx=0 tx=0).stop 2024-04-26T22:01:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 msgr2=0x7ff26c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 0x7ff26c19b810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e9f0 tx=0x7ff25c00ed00 comp rx=0 tx=0).stop 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 shutdown_connections 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff2500a3cd0 0x7ff2500a6190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff26c1030a0 0x7ff26c1a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c101d80 0x7ff26c19b810 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 --2- 172.21.15.12:0/3274405408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff26c1026e0 0x7ff26c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.911+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 >> 172.21.15.12:0/3274405408 conn(0x7ff26c0fd920 msgr2=0x7ff26c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.912+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 shutdown_connections 2024-04-26T22:01:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:54.912+0000 7ff272800700 1 -- 172.21.15.12:0/3274405408 wait complete. 2024-04-26T22:01:55.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:54 smithi077 ceph-mon[32280]: pgmap v389: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 879 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:55.289 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:54 smithi012 ceph-mon[28184]: pgmap v389: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 879 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:55.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:54 smithi154 ceph-mon[32578]: pgmap v389: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 879 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:55.717 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T22:01:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:01:56.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:55 smithi077 ceph-mon[32280]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:56.292 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:55 smithi154 ceph-mon[32578]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:56.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:55 smithi012 ceph-mon[28184]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:57.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:56 smithi077 ceph-mon[32280]: pgmap v390: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:57.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:56 smithi012 ceph-mon[28184]: pgmap v390: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:57.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:56 smithi154 ceph-mon[32578]: pgmap v390: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:58.213 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:57 smithi012 ceph-mon[28184]: pgmap v391: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:58.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:57 smithi077 ceph-mon[32280]: pgmap v391: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:58.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 -- 172.21.15.12:0/1488803517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec109170 msgr2=0x7f7fec10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:58.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/1488803517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec109170 0x7f7fec10b5e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b0a0 tx=0x7f7fe400b3b0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 -- 172.21.15.12:0/1488803517 shutdown_connections 2024-04-26T22:01:58.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/1488803517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec109170 0x7f7fec10b5e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/1488803517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/1488803517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec106840 0x7f7fec108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 -- 172.21.15.12:0/1488803517 >> 172.21.15.12:0/1488803517 conn(0x7f7fec0fd940 msgr2=0x7f7fec0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 -- 172.21.15.12:0/1488803517 shutdown_connections 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.388+0000 7f7ff0a00700 1 -- 172.21.15.12:0/1488803517 wait complete. 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.389+0000 7f7ff0a00700 1 Processor -- start 2024-04-26T22:01:58.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 -- start start 2024-04-26T22:01:58.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 0x7f7fec19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 0x7f7fec1a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10ef00 con 0x7f7fec103f10 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10ed80 con 0x7f7fec109170 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7ff0a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10f080 con 0x7f7fec106840 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.390+0000 7f7fe9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fe9e00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44212/0 (socket says 172.21.15.12:44212) 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fe9e00700 1 -- 172.21.15.12:0/3370280423 learned_addr learned my addr 172.21.15.12:0/3370280423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 0x7f7fec1a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:58.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fe9400700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 0x7f7fec19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:58.392 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:57 smithi154 ceph-mon[32578]: pgmap v391: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 -- 172.21.15.12:0/3370280423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 msgr2=0x7f7fec19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 0x7f7fec19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 -- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 msgr2=0x7f7fec19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fea800700 1 -- 172.21.15.12:0/3370280423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1a33e0 con 0x7f7fec109170 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fe9e00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.391+0000 7f7fe9400700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 0x7f7fec19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7fea800700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 0x7f7fec1a2d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4006130 tx=0x7f7fe4004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4021070 con 0x7f7fec109170 2024-04-26T22:01:58.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe4007b80 con 0x7f7fec109170 2024-04-26T22:01:58.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec1a3670 con 0x7f7fec109170 2024-04-26T22:01:58.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4005040 con 0x7f7fec109170 2024-04-26T22:01:58.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.392+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a3c00 con 0x7f7fec109170 2024-04-26T22:01:58.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.393+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec06a110 con 0x7f7fec109170 2024-04-26T22:01:58.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.395+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7fe4003a00 con 0x7f7fec109170 2024-04-26T22:01:58.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.397+0000 7f7fe2a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 0x7f7fd00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:58.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.397+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7fe400f020 con 0x7f7fec109170 2024-04-26T22:01:58.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.397+0000 7f7fe9e00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 0x7f7fd00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:58.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.398+0000 7f7fe9e00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 0x7f7fd00a6210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011550 tx=0x7f7fdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:58.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.399+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7fe402a030 con 0x7f7fec109170 2024-04-26T22:01:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.641+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7fec063860 con 0x7f7fd00a3d50 2024-04-26T22:01:58.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.672+0000 7f7fe2a00700 1 -- 172.21.15.12:0/3370280423 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7fec063860 con 0x7f7fd00a3d50 2024-04-26T22:01:58.672 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (2m) 33s ago 2m 12.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7993b81c1ce6 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi077 running (2m) 72s ago 2m 25.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 26be940ac9ec 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi154 running (2m) 72s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 0b215deba1c0 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (2m) 33s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 738205e92810 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi077 running (2m) 72s ago 2m 14.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fcec7ef69cb3 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi154 running (2m) 72s ago 2m 16.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 295891c0a8b1 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (2m) 33s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 38277650be1c 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi077 running (2m) 72s ago 2m 13.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 04a979fe9e63 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi154 running (2m) 72s ago 2m 14.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 96dfd8e6f8ec 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (2m) 33s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 e549a528f486 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi077 running (2m) 72s ago 2m 14.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 339bd94210aa 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi154 running (2m) 72s ago 2m 16.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 63fb37f88206 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (11m) 33s ago 11m 515M - 19.0.0-3306-gbc612ffb 55dc54019ad8 0bfdf66e9f03 2024-04-26T22:01:58.673 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi077 *:8443,8765 running (8m) 72s ago 8m 455M - 19.0.0-3306-gbc612ffb 55dc54019ad8 23cae74151fb 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi154 *:8443,8765 running (8m) 72s ago 8m 453M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d1d8d36241ea 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (11m) 33s ago 11m 48.5M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 057f9fa44791 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi077 running (9m) 72s ago 9m 41.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 d7a1f61a5efe 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi154 running (9m) 72s ago 9m 40.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 e63615a5e89d 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (8m) 33s ago 8m 46.9M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 4086721f5076 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi077 running (8m) 72s ago 8m 54.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 d73567eb1d71 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi154 running (7m) 72s ago 7m 57.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 00162dbed153 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (7m) 33s ago 7m 43.4M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 531b0f14cc75 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi077 running (6m) 72s ago 6m 42.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 377cc377708a 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi154 running (6m) 72s ago 6m 52.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 3173c977f3d2 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (5m) 33s ago 5m 41.4M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 1e36a681767b 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi077 running (5m) 72s ago 5m 52.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 73568df389df 2024-04-26T22:01:58.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi154 running (4m) 72s ago 4m 43.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 cffb63664391 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (4m) 33s ago 4m 44.1M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 df0a3ab05fcd 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi077 running (3m) 72s ago 3m 53.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b087d687d933 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi154 running (3m) 72s ago 3m 52.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4242eeda4879 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 msgr2=0x7f7fd00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 0x7f7fd00a6210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc011550 tx=0x7f7fdc009260 comp rx=0 tx=0).stop 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 msgr2=0x7f7fec1a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 0x7f7fec1a2d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4006130 tx=0x7f7fe4004ce0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 shutdown_connections 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fec109170 0x7f7fec1a2d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fec103f10 0x7f7fec19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7fd00a3d50 0x7f7fd00a6210 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 --2- 172.21.15.12:0/3370280423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fec106840 0x7f7fec19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 >> 172.21.15.12:0/3370280423 conn(0x7f7fec0fd940 msgr2=0x7f7fec103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 shutdown_connections 2024-04-26T22:01:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:01:58.675+0000 7f7ff0a00700 1 -- 172.21.15.12:0/3370280423 wait complete. 2024-04-26T22:01:59.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:58 smithi077 ceph-mon[32280]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:59.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:58 smithi012 ceph-mon[28184]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:59.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:58 smithi154 ceph-mon[32578]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:01:59.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T22:02:00.046 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:00.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:01:59 smithi077 ceph-mon[32280]: pgmap v392: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:00.236 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:01:59 smithi012 ceph-mon[28184]: pgmap v392: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:00.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:01:59 smithi154 ceph-mon[32578]: pgmap v392: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 -- 172.21.15.12:0/3625571855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 msgr2=0x7f174810b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 --2- 172.21.15.12:0/3625571855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f174810b5e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1740009990 tx=0x7f1740009ca0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 -- 172.21.15.12:0/3625571855 shutdown_connections 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 --2- 172.21.15.12:0/3625571855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f174810b5e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 --2- 172.21.15.12:0/3625571855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1748103f70 0x7f1748106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 --2- 172.21.15.12:0/3625571855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17481068a0 0x7f1748108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 -- 172.21.15.12:0/3625571855 >> 172.21.15.12:0/3625571855 conn(0x7f17480fd920 msgr2=0x7f17480ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 -- 172.21.15.12:0/3625571855 shutdown_connections 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.249+0000 7f174ca00700 1 -- 172.21.15.12:0/3625571855 wait complete. 2024-04-26T22:02:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.250+0000 7f174ca00700 1 Processor -- start 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 -- start start 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17481068a0 0x7f1748101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f1748102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f174810ef50 con 0x7f17481091d0 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f174810edd0 con 0x7f1748103f70 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.251+0000 7f174ca00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f174810f0d0 con 0x7f17481068a0 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f1748102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1745e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1745e00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:55988/0 (socket says 172.21.15.12:55988) 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f173d400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17481068a0 0x7f1748101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1745e00700 1 -- 172.21.15.12:0/701928119 learned_addr learned my addr 172.21.15.12:0/701928119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 -- 172.21.15.12:0/701928119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17481068a0 msgr2=0x7f1748101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17481068a0 0x7f1748101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 -- 172.21.15.12:0/701928119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 msgr2=0x7f1748103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 -- 172.21.15.12:0/701928119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1748102d30 con 0x7f17481091d0 2024-04-26T22:02:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1745e00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.252+0000 7f1746800700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f1748102460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f17400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.253+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740026070 con 0x7f17481091d0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.253+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1740021410 con 0x7f17481091d0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.253+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740017650 con 0x7f17481091d0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.253+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17481a3730 con 0x7f17481091d0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.253+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17481a3b90 con 0x7f17481091d0 2024-04-26T22:02:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.254+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174806a110 con 0x7f17481091d0 2024-04-26T22:02:02.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.256+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1740029070 con 0x7f17481091d0 2024-04-26T22:02:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.257+0000 7f173f400700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 0x7f172c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:02.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.258+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f17400d15a0 con 0x7f17481091d0 2024-04-26T22:02:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.258+0000 7f1745e00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 0x7f172c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:02.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.259+0000 7f1745e00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 0x7f172c0a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1738011590 tx=0x7f1738009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:02.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.262+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f174005e060 con 0x7f17481091d0 2024-04-26T22:02:02.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.503+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1748063860 con 0x7f172c0a3d50 2024-04-26T22:02:02.504 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:02 smithi012 ceph-mon[28184]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.517+0000 7f173f400700 1 -- 172.21.15.12:0/701928119 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1748063860 con 0x7f172c0a3d50 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 76s ago 3m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 76s ago 9m smithi012=x;smithi077=y;smithi154=z;count:3 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 76s ago 9m smithi012:172.21.15.12=a;smithi077:172.21.15.77=b;smithi154:172.21.15.154=c;count:3 2024-04-26T22:02:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 76s ago - 2024-04-26T22:02:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 msgr2=0x7f172c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 0x7f172c0a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1738011590 tx=0x7f1738009260 comp rx=0 tx=0).stop 2024-04-26T22:02:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 msgr2=0x7f1748102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f1748102460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f17400057c0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 shutdown_connections 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17481091d0 0x7f1748102460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1748103f70 0x7f1748103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f172c0a3d50 0x7f172c0a6210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 --2- 172.21.15.12:0/701928119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17481068a0 0x7f1748101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 >> 172.21.15.12:0/701928119 conn(0x7f17480fd920 msgr2=0x7f17481048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.520+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 shutdown_connections 2024-04-26T22:02:02.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:02.521+0000 7f174ca00700 1 -- 172.21.15.12:0/701928119 wait complete. 2024-04-26T22:02:02.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:02 smithi154 ceph-mon[32578]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:02.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:02 smithi077 ceph-mon[32280]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:03.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:03 smithi012 ceph-mon[28184]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:03.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T22:02:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:03 smithi154 ceph-mon[32578]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:03 smithi077 ceph-mon[32280]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:03.825 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:04.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:04 smithi012 ceph-mon[28184]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:04.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:04 smithi154 ceph-mon[32578]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:04.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:04 smithi077 ceph-mon[32280]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.035+0000 7f4cb4200700 1 -- 172.21.15.12:0/3067401176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 msgr2=0x7f4cac104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.035+0000 7f4cb4200700 1 --2- 172.21.15.12:0/3067401176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac104ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009920 tx=0x7f4c98009c30 comp rx=0 tx=0).stop 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 -- 172.21.15.12:0/3067401176 shutdown_connections 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 --2- 172.21.15.12:0/3067401176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 0x7f4cac10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 --2- 172.21.15.12:0/3067401176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 --2- 172.21.15.12:0/3067401176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac104ae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 -- 172.21.15.12:0/3067401176 >> 172.21.15.12:0/3067401176 conn(0x7f4cac0fea20 msgr2=0x7f4cac100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:06.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 -- 172.21.15.12:0/3067401176 shutdown_connections 2024-04-26T22:02:06.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.036+0000 7f4cb4200700 1 -- 172.21.15.12:0/3067401176 wait complete. 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 Processor -- start 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 -- start start 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 0x7f4cac1a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac10ef00 con 0x7f4cac105020 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac10ed80 con 0x7f4cac104680 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.037+0000 7f4cb4200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac10f080 con 0x7f4cac103480 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb2200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 0x7f4cac1a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:35588/0 (socket says 172.21.15.12:35588) 2024-04-26T22:02:06.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 -- 172.21.15.12:0/771530517 learned_addr learned my addr 172.21.15.12:0/771530517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb0e00700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 -- 172.21.15.12:0/771530517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 msgr2=0x7f4cac19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 -- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 msgr2=0x7f4cac1a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 0x7f4cac1a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb1800700 1 -- 172.21.15.12:0/771530517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1a34b0 con 0x7f4cac103480 2024-04-26T22:02:06.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.038+0000 7f4cb0e00700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4cb1800700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000ea20 tx=0x7f4ca000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0004220 con 0x7f4cac103480 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca00043c0 con 0x7f4cac103480 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0003bb0 con 0x7f4cac103480 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac1a37a0 con 0x7f4cac103480 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.039+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a3c90 con 0x7f4cac103480 2024-04-26T22:02:06.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.040+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac06a110 con 0x7f4cac103480 2024-04-26T22:02:06.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.043+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ca000c7c0 con 0x7f4cac103480 2024-04-26T22:02:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.044+0000 7f4caa000700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 0x7f4c940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.044+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ca00d0b50 con 0x7f4cac103480 2024-04-26T22:02:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.044+0000 7f4cb0e00700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 0x7f4c940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:06.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.045+0000 7f4cb0e00700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 0x7f4c940a6210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800faf0 tx=0x7f4c98019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:06.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.046+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4ca009d060 con 0x7f4cac103480 2024-04-26T22:02:06.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.289+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4cac10b5c0 con 0x7f4c940a3d50 2024-04-26T22:02:06.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.292+0000 7f4caa000700 1 -- 172.21.15.12:0/771530517 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4cac10b5c0 con 0x7f4c940a3d50 2024-04-26T22:02:06.294 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-04-26T22:02:06.294 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-04-26T22:02:06.294 INFO:teuthology.orchestra.run.smithi012.stdout:smithi077 172.21.15.77 2024-04-26T22:02:06.294 INFO:teuthology.orchestra.run.smithi012.stdout:smithi154 172.21.15.154 2024-04-26T22:02:06.294 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-04-26T22:02:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 msgr2=0x7f4c940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 0x7f4c940a6210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800faf0 tx=0x7f4c98019040 comp rx=0 tx=0).stop 2024-04-26T22:02:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 msgr2=0x7f4cac19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000ea20 tx=0x7f4ca000ed30 comp rx=0 tx=0).stop 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 shutdown_connections 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cac105020 0x7f4cac1a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f4c940a3d50 0x7f4c940a6210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cac103480 0x7f4cac19b850 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 --2- 172.21.15.12:0/771530517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cac104680 0x7f4cac19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 >> 172.21.15.12:0/771530517 conn(0x7f4cac0fea20 msgr2=0x7f4cac109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.295+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 shutdown_connections 2024-04-26T22:02:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:06.296+0000 7f4cb4200700 1 -- 172.21.15.12:0/771530517 wait complete. 2024-04-26T22:02:06.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:06 smithi012 ceph-mon[28184]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:06.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:06 smithi154 ceph-mon[32578]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:06.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:06 smithi077 ceph-mon[32280]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:07.112 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T22:02:07.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:07 smithi012 ceph-mon[28184]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:07.594 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:07 smithi154 ceph-mon[32578]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:07.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:07 smithi077 ceph-mon[32280]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:08.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:08 smithi012 ceph-mon[28184]: pgmap v396: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:08.633 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:08 smithi077 ceph-mon[32280]: pgmap v396: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:08.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:08 smithi154 ceph-mon[32578]: pgmap v396: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 -- 172.21.15.12:0/3373919769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810b660 msgr2=0x7f26e8111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 --2- 172.21.15.12:0/3373919769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810b660 0x7f26e8111f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009fc0 tx=0x7f26e000b330 comp rx=0 tx=0).stop 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 -- 172.21.15.12:0/3373919769 shutdown_connections 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 --2- 172.21.15.12:0/3373919769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810b660 0x7f26e8111f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 --2- 172.21.15.12:0/3373919769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e8109ac0 0x7f26e8109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 --2- 172.21.15.12:0/3373919769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e810acc0 0x7f26e810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 -- 172.21.15.12:0/3373919769 >> 172.21.15.12:0/3373919769 conn(0x7f26e8078a90 msgr2=0x7f26e807aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 -- 172.21.15.12:0/3373919769 shutdown_connections 2024-04-26T22:02:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.874+0000 7f26ee200700 1 -- 172.21.15.12:0/3373919769 wait complete. 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.876+0000 7f26ee200700 1 Processor -- start 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.876+0000 7f26ee200700 1 -- start start 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.876+0000 7f26ee200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e8109ac0 0x7f26e819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.876+0000 7f26ee200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810acc0 0x7f26e81a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.876+0000 7f26ee200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26ee200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8115540 con 0x7f26e810acc0 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26ee200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81153c0 con 0x7f26e8109ac0 2024-04-26T22:02:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26ee200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f26e81156c0 con 0x7f26e810b660 2024-04-26T22:02:09.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:09.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7400700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e8109ac0 0x7f26e819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:09.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26dfe00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810acc0 0x7f26e81a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43764/0 (socket says 172.21.15.12:43764) 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 -- 172.21.15.12:0/1774250611 learned_addr learned my addr 172.21.15.12:0/1774250611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 -- 172.21.15.12:0/1774250611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e8109ac0 msgr2=0x7f26e819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e8109ac0 0x7f26e819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 -- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810acc0 msgr2=0x7f26e81a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810acc0 0x7f26e81a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.877+0000 7f26e7e00700 1 -- 172.21.15.12:0/1774250611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e81a7790 con 0x7f26e810b660 2024-04-26T22:02:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26e7e00700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26e000cfd0 tx=0x7f26e0004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0021070 con 0x7f26e810b660 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e0007ac0 con 0x7f26e810b660 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81a7a20 con 0x7f26e810b660 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00052a0 con 0x7f26e810b660 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.878+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81a7ed0 con 0x7f26e810b660 2024-04-26T22:02:09.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.880+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e806a110 con 0x7f26e810b660 2024-04-26T22:02:09.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.882+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f26e0024070 con 0x7f26e810b660 2024-04-26T22:02:09.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.883+0000 7f26e4c00700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 0x7f26cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.883+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f26e000f020 con 0x7f26e810b660 2024-04-26T22:02:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.883+0000 7f26e7400700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 0x7f26cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:09.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.884+0000 7f26e7400700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 0x7f26cc0a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009750 tx=0x7f26d8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:09.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:09.887+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26e002b020 con 0x7f26e810b660 2024-04-26T22:02:10.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.126+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f26e8063860 con 0x7f26cc0a3cd0 2024-04-26T22:02:10.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.129+0000 7f26e4c00700 1 -- 172.21.15.12:0/1774250611 <== mgr.14152 v2:172.21.15.12:6800/3660687894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f26e8063860 con 0x7f26cc0a3cd0 2024-04-26T22:02:10.130 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T22:02:10.130 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T22:02:10.130 INFO:teuthology.orchestra.run.smithi012.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T22:02:10.130 INFO:teuthology.orchestra.run.smithi012.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T22:02:10.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.131+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 msgr2=0x7f26cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:10.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.131+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 0x7f26cc0a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009750 tx=0x7f26d8009310 comp rx=0 tx=0).stop 2024-04-26T22:02:10.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 msgr2=0x7f26e81a70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:10.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26e000cfd0 tx=0x7f26e0004f00 comp rx=0 tx=0).stop 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 shutdown_connections 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f26cc0a3cd0 0x7f26cc0a6190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26e810b660 0x7f26e81a70f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e8109ac0 0x7f26e819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 --2- 172.21.15.12:0/1774250611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26e810acc0 0x7f26e81a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:10.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 >> 172.21.15.12:0/1774250611 conn(0x7f26e8078a90 msgr2=0x7f26e810dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:10.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 shutdown_connections 2024-04-26T22:02:10.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:10.132+0000 7f26ee200700 1 -- 172.21.15.12:0/1774250611 wait complete. 2024-04-26T22:02:10.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:10 smithi012 ceph-mon[28184]: pgmap v397: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:10.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:10 smithi154 ceph-mon[32578]: pgmap v397: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:10.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:10 smithi077 ceph-mon[32280]: pgmap v397: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:10.953 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T22:02:10.969 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:02:10.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T22:02:11.451 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:11 smithi012 ceph-mon[28184]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:11 smithi154 ceph-mon[32578]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:11 smithi077 ceph-mon[32280]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:12.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:12 smithi012 ceph-mon[28184]: pgmap v398: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:12.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:12 smithi154 ceph-mon[32578]: pgmap v398: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:12.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:12 smithi077 ceph-mon[32280]: pgmap v398: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 -- 172.21.15.12:0/1206155998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c1034b0 msgr2=0x7fb50c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 --2- 172.21.15.12:0/1206155998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c1034b0 0x7fb50c1038d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0099c0 tx=0x7fb4fc009cd0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 -- 172.21.15.12:0/1206155998 shutdown_connections 2024-04-26T22:02:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 --2- 172.21.15.12:0/1206155998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c105050 0x7fb50c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 --2- 172.21.15.12:0/1206155998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c1034b0 0x7fb50c1038d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 --2- 172.21.15.12:0/1206155998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 -- 172.21.15.12:0/1206155998 >> 172.21.15.12:0/1206155998 conn(0x7fb50c0fea70 msgr2=0x7fb50c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 -- 172.21.15.12:0/1206155998 shutdown_connections 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.844+0000 7fb512000700 1 -- 172.21.15.12:0/1206155998 wait complete. 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.846+0000 7fb512000700 1 Processor -- start 2024-04-26T22:02:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.846+0000 7fb512000700 1 -- start start 2024-04-26T22:02:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 0x7fb50c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:13.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10ef90 con 0x7fb50c1046b0 2024-04-26T22:02:13.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10ee10 con 0x7fb50c105050 2024-04-26T22:02:13.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb512000700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10f110 con 0x7fb50c1034b0 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50b400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 0x7fb50c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50be00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50be00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38202/0 (socket says 172.21.15.12:38202) 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50be00700 1 -- 172.21.15.12:0/4241390187 learned_addr learned my addr 172.21.15.12:0/4241390187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50aa00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33132/0 (socket says 172.21.15.12:33132) 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.847+0000 7fb50b400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 0x7fb50c199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43778/0 (socket says 172.21.15.12:43778) 2024-04-26T22:02:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 -- 172.21.15.12:0/4241390187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 msgr2=0x7fb50c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 0x7fb50c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 -- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 msgr2=0x7fb50c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 -- 172.21.15.12:0/4241390187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c1a1260 con 0x7fb50c105050 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb50be00700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb50400ef30 tx=0x7fb50400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5040092e0 con 0x7fb50c105050 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c1a14f0 con 0x7fb50c105050 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb504004bb0 con 0x7fb50c105050 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504004020 con 0x7fb50c105050 2024-04-26T22:02:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.848+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a19e0 con 0x7fb50c105050 2024-04-26T22:02:13.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.850+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c06a110 con 0x7fb50c105050 2024-04-26T22:02:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.852+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb50401b030 con 0x7fb50c105050 2024-04-26T22:02:13.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.853+0000 7fb503e00700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 0x7fb4f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:13.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.853+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb5040d0370 con 0x7fb50c105050 2024-04-26T22:02:13.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.853+0000 7fb50b400700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 0x7fb4f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.854+0000 7fb50b400700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 0x7fb4f00a6190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00b580 tx=0x7fb4fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:13.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:13.857+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb504022020 con 0x7fb50c105050 2024-04-26T22:02:14.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.140+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb50c02e4e0 con 0x7fb50c105050 2024-04-26T22:02:14.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.141+0000 7fb503e00700 1 -- 172.21.15.12:0/4241390187 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3004 (secure 0 0 0) 0x7fb504010040 con 0x7fb50c105050 2024-04-26T22:02:14.143 INFO:teuthology.orchestra.run.smithi012.stdout:e21 2024-04-26T22:02:14.143 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T22:02:14.143 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:02:14.143 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 21 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-26T21:59:43.592535+0000 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-04-26T22:00:09.736381+0000 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-04-26T22:02:14.144 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 1 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:in 0 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24581} 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3,4] 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 24581 members: 24581 2024-04-26T22:02:14.145 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{0:24565} state up:standby-replay seq 1 addr [v2:172.21.15.12:6838/3734088968,v1:172.21.15.12:6839/3734088968] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.12:6834/2930574103,v1:172.21.15.12:6835/2930574103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14400} state up:standby seq 1 addr [v2:172.21.15.12:6836/247823794,v1:172.21.15.12:6837/247823794] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{-1:14472} state up:standby seq 1 addr [v2:172.21.15.12:6840/1291467894,v1:172.21.15.12:6841/1291467894] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.77:6832/432973357,v1:172.21.15.77:6833/432973357] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.154:6832/1839140447,v1:172.21.15.154:6833/1839140447] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{-1:24497} state up:standby seq 1 addr [v2:172.21.15.77:6834/3868639549,v1:172.21.15.77:6835/3868639549] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{-1:24514} state up:standby seq 1 addr [v2:172.21.15.154:6834/1762309874,v1:172.21.15.154:6835/1762309874] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{-1:24524} state up:standby seq 1 addr [v2:172.21.15.77:6836/1703155205,v1:172.21.15.77:6837/1703155205] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.146 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{-1:24526} state up:standby seq 1 addr [v2:172.21.15.154:6836/175106835,v1:172.21.15.154:6837/175106835] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{-1:24553} state up:standby seq 1 addr [v2:172.21.15.154:6838/404103825,v1:172.21.15.154:6839/404103825] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T22:02:14.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.145+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 msgr2=0x7fb4f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:14.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.145+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 0x7fb4f00a6190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00b580 tx=0x7fb4fc005fb0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.145+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 msgr2=0x7fb50c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.145+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb50400ef30 tx=0x7fb50400c3a0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 shutdown_connections 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c105050 0x7fb50c1a0bc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb4f00a3cd0 0x7fb4f00a6190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb50c1034b0 0x7fb50c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 --2- 172.21.15.12:0/4241390187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb50c1046b0 0x7fb50c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 >> 172.21.15.12:0/4241390187 conn(0x7fb50c0fea70 msgr2=0x7fb50c107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 shutdown_connections 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:14.146+0000 7fb512000700 1 -- 172.21.15.12:0/4241390187 wait complete. 2024-04-26T22:02:14.148 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:02:14.396 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:14 smithi012 ceph-mon[28184]: pgmap v399: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:14.396 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:14 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4241390187' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:02:14.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:14 smithi154 ceph-mon[32578]: pgmap v399: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:14.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:14 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4241390187' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:02:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:14 smithi077 ceph-mon[32280]: pgmap v399: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:14.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:14 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4241390187' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T22:02:14.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T22:02:15.468 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:16.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:16 smithi012 ceph-mon[28184]: pgmap v400: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:16.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:16 smithi154 ceph-mon[32578]: pgmap v400: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:16.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:16 smithi077 ceph-mon[32280]: pgmap v400: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.775+0000 7f6f80200700 1 -- 172.21.15.12:0/1327268893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 msgr2=0x7f6f7810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.775+0000 7f6f80200700 1 --2- 172.21.15.12:0/1327268893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f7810db10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000b0d0 tx=0x7f6f7000b3e0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 -- 172.21.15.12:0/1327268893 shutdown_connections 2024-04-26T22:02:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 --2- 172.21.15.12:0/1327268893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f7810db10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 --2- 172.21.15.12:0/1327268893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f78105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 --2- 172.21.15.12:0/1327268893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 0x7f6f78106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 -- 172.21.15.12:0/1327268893 >> 172.21.15.12:0/1327268893 conn(0x7f6f78100c80 msgr2=0x7f6f781030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 -- 172.21.15.12:0/1327268893 shutdown_connections 2024-04-26T22:02:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.776+0000 7f6f80200700 1 -- 172.21.15.12:0/1327268893 wait complete. 2024-04-26T22:02:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 Processor -- start 2024-04-26T22:02:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 -- start start 2024-04-26T22:02:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 0x7f6f7819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f781a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111190 con 0x7f6f78107260 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111010 con 0x7f6f781056c0 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.777+0000 7f6f80200700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111310 con 0x7f6f781068c0 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7e200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f781a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7ce00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 0x7f6f7819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38214/0 (socket says 172.21.15.12:38214) 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7e200700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f781a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33142/0 (socket says 172.21.15.12:33142) 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 -- 172.21.15.12:0/3820598388 learned_addr learned my addr 172.21.15.12:0/3820598388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 -- 172.21.15.12:0/3820598388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 msgr2=0x7f6f7819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 0x7f6f7819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 -- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 msgr2=0x7f6f781a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f781a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.778+0000 7f6f7d800700 1 -- 172.21.15.12:0/3820598388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f781a3350 con 0x7f6f781056c0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f7d800700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00e970 tx=0x7f6f6c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c004220 con 0x7f6f781056c0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f781a3640 con 0x7f6f781056c0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6c0043c0 con 0x7f6f781056c0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c003ce0 con 0x7f6f781056c0 2024-04-26T22:02:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.779+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781a3b80 con 0x7f6f781056c0 2024-04-26T22:02:17.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.780+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7806a110 con 0x7f6f781056c0 2024-04-26T22:02:17.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.782+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f6c00c7e0 con 0x7f6f781056c0 2024-04-26T22:02:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.784+0000 7f6f76000700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 0x7f6f600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.784+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6f6c0d0d20 con 0x7f6f781056c0 2024-04-26T22:02:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.784+0000 7f6f7ce00700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 0x7f6f600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.784+0000 7f6f7ce00700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 0x7f6f600a6190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009680 tx=0x7f6f64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:17.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:17.786+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f6c09d1a0 con 0x7f6f781056c0 2024-04-26T22:02:18.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.013+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6f7802e510 con 0x7f6f781056c0 2024-04-26T22:02:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.015+0000 7f6f76000700 1 -- 172.21.15.12:0/3820598388 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5646 (secure 0 0 0) 0x7f6f6c09caa0 con 0x7f6f781056c0 2024-04-26T22:02:18.016 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 94 2024-04-26T22:02:18.016 INFO:teuthology.orchestra.run.smithi012.stdout:fsid d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:02:18.016 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-26T21:50:26.382191+0000 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-04-26T22:01:44.865520+0000 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 27 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T22:02:18.017 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/1061847783,v1:172.21.15.12:6803/1061847783] [v2:172.21.15.12:6804/1061847783,v1:172.21.15.12:6805/1061847783] exists,up 26c2f4bf-8ef4-4f89-9dc8-b00f6e556980 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/828320390,v1:172.21.15.77:6801/828320390] [v2:172.21.15.77:6802/828320390,v1:172.21.15.77:6803/828320390] exists,up f493e866-faee-4d6e-a462-c582971cf31b 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/3122867729,v1:172.21.15.154:6801/3122867729] [v2:172.21.15.154:6802/3122867729,v1:172.21.15.154:6803/3122867729] exists,up 1a38e236-f013-4ab3-9310-272f5ec1b653 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/1602812044,v1:172.21.15.12:6811/1602812044] [v2:172.21.15.12:6812/1602812044,v1:172.21.15.12:6813/1602812044] exists,up d44a553b-4160-4233-b691-acd3104b7eac 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/482924396,v1:172.21.15.77:6809/482924396] [v2:172.21.15.77:6810/482924396,v1:172.21.15.77:6811/482924396] exists,up 13cc095e-3560-47a1-bca1-4dc3cbbc8198 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/1747631146,v1:172.21.15.154:6809/1747631146] [v2:172.21.15.154:6810/1747631146,v1:172.21.15.154:6811/1747631146] exists,up 38bc196f-d165-47ad-815e-da841218d3a2 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 47 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3229053639,v1:172.21.15.12:6819/3229053639] [v2:172.21.15.12:6820/3229053639,v1:172.21.15.12:6821/3229053639] exists,up b5475e1d-8f75-4b95-b8d5-45916e0ed62f 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/1953584019,v1:172.21.15.77:6817/1953584019] [v2:172.21.15.77:6818/1953584019,v1:172.21.15.77:6819/1953584019] exists,up 00d199a4-d87f-4184-b154-13d6f3dd67e9 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/165436981,v1:172.21.15.154:6817/165436981] [v2:172.21.15.154:6818/165436981,v1:172.21.15.154:6819/165436981] exists,up 03a5cda5-c502-4af5-88e3-3196bf23380d 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/1774118135,v1:172.21.15.12:6827/1774118135] [v2:172.21.15.12:6828/1774118135,v1:172.21.15.12:6829/1774118135] exists,up 70375e11-a2c2-4512-9403-6cc31fb3409c 2024-04-26T22:02:18.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 73 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6824/1999551212,v1:172.21.15.77:6825/1999551212] [v2:172.21.15.77:6826/1999551212,v1:172.21.15.77:6827/1999551212] exists,up 1a526fb3-2f0b-4161-9b40-9ba2774e847a 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 81 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/3735995380,v1:172.21.15.154:6825/3735995380] [v2:172.21.15.154:6826/3735995380,v1:172.21.15.154:6827/3735995380] exists,up bfad9b30-8b56-4b29-9722-cf3de267af70 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.12 [5,11] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.1b [2,4] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.1f [8,1] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.10 [5,9] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.24 [4,9] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.31 [4,8] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.3c [4,9] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.5 [9,10] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.c [9,10] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.15 [5,10] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.1b [9,8] 2024-04-26T22:02:18.019 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/2513368485 expires 2024-04-27T21:51:42.978579+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/2513368485 expires 2024-04-27T21:51:42.978579+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3243088974 expires 2024-04-27T21:51:07.889475+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1310858189 expires 2024-04-27T21:51:07.889475+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/1377681366 expires 2024-04-27T21:51:07.889475+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/1377681366 expires 2024-04-27T21:51:07.889475+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3387900079 expires 2024-04-27T21:51:42.978579+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/600788188 expires 2024-04-27T21:51:07.889475+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2638555806 expires 2024-04-27T21:51:42.978579+0000 2024-04-26T22:02:18.020 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4197187136 expires 2024-04-27T21:51:42.978579+0000 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 msgr2=0x7f6f600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 0x7f6f600a6190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009680 tx=0x7f6f64009260 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 msgr2=0x7f6f7819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00e970 tx=0x7f6f6c00ec80 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 shutdown_connections 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f78107260 0x7f6f781a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f781056c0 0x7f6f7819b6f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f6f600a3cd0 0x7f6f600a6190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 --2- 172.21.15.12:0/3820598388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f781068c0 0x7f6f7819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 >> 172.21.15.12:0/3820598388 conn(0x7f6f78100c80 msgr2=0x7f6f78109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 shutdown_connections 2024-04-26T22:02:18.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:18.018+0000 7f6f80200700 1 -- 172.21.15.12:0/3820598388 wait complete. 2024-04-26T22:02:18.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:18 smithi012 ceph-mon[28184]: pgmap v401: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:18.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:18 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3820598388' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:02:18.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:18 smithi154 ceph-mon[32578]: pgmap v401: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:18.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:18 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3820598388' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:02:18.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:18 smithi077 ceph-mon[32280]: pgmap v401: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:18.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:18 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3820598388' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T22:02:18.935 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T22:02:18.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 -- 172.21.15.12:0/3165902217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 msgr2=0x7fb844104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 --2- 172.21.15.12:0/3165902217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb844104db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb834009ca0 tx=0x7fb83402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 -- 172.21.15.12:0/3165902217 shutdown_connections 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 --2- 172.21.15.12:0/3165902217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 0x7fb84410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 --2- 172.21.15.12:0/3165902217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb844106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 --2- 172.21.15.12:0/3165902217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb844104db0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 -- 172.21.15.12:0/3165902217 >> 172.21.15.12:0/3165902217 conn(0x7fb8440fff00 msgr2=0x7fb844102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:19.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 -- 172.21.15.12:0/3165902217 shutdown_connections 2024-04-26T22:02:19.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.179+0000 7fb849000640 1 -- 172.21.15.12:0/3165902217 wait complete. 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.180+0000 7fb849000640 1 Processor -- start 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.180+0000 7fb849000640 1 -- start start 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.180+0000 7fb849000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb84419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb849000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb84419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb849000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 0x7fb8441a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb849000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8441104e0 con 0x7fb844104990 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb849000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb844110360 con 0x7fb844106540 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb849000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb844110660 con 0x7fb844105b80 2024-04-26T22:02:19.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb843400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb84419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb843e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 0x7fb8441a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb843400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb84419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34336/0 (socket says 172.21.15.12:34336) 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb843400640 1 -- 172.21.15.12:0/4019119713 learned_addr learned my addr 172.21.15.12:0/4019119713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb84419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 -- 172.21.15.12:0/4019119713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 msgr2=0x7fb8441a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 0x7fb8441a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 -- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 msgr2=0x7fb84419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb84419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.181+0000 7fb842a00640 1 -- 172.21.15.12:0/4019119713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8441a2860 con 0x7fb844105b80 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb842a00640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb84419b190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e9f0 tx=0x7fb82c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00ccc0 con 0x7fb844105b80 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c0108e0 con 0x7fb844105b80 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c018840 con 0x7fb844105b80 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441a2b50 con 0x7fb844105b80 2024-04-26T22:02:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.182+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441a3050 con 0x7fb844105b80 2024-04-26T22:02:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.183+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84410bdd0 con 0x7fb844105b80 2024-04-26T22:02:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.186+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb82c010430 con 0x7fb844105b80 2024-04-26T22:02:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.187+0000 7fb83be00640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 0x7fb8280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.188+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb82c0d0bf0 con 0x7fb844105b80 2024-04-26T22:02:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.188+0000 7fb843400640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 0x7fb8280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.188+0000 7fb843400640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 0x7fb8280a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb834031b50 tx=0x7fb834009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:19.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.189+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb82c09d070 con 0x7fb844105b80 2024-04-26T22:02:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.395+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb84402dd40 con 0x7fb844105b80 2024-04-26T22:02:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.398+0000 7fb83be00640 1 -- 172.21.15.12:0/4019119713 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7fb82c09c970 con 0x7fb844105b80 2024-04-26T22:02:19.399 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:19.399 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 msgr2=0x7fb8280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 0x7fb8280a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb834031b50 tx=0x7fb834009880 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 msgr2=0x7fb84419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb84419b190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e9f0 tx=0x7fb82c00edf0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 shutdown_connections 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fb8280a3cc0 0x7fb8280a6180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb844106540 0x7fb8441a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb844105b80 0x7fb84419b190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 --2- 172.21.15.12:0/4019119713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb844104990 0x7fb84419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 >> 172.21.15.12:0/4019119713 conn(0x7fb8440fff00 msgr2=0x7fb844108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 shutdown_connections 2024-04-26T22:02:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.401+0000 7fb849000640 1 -- 172.21.15.12:0/4019119713 wait complete. 2024-04-26T22:02:19.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:02:19.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:02:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 -- 172.21.15.12:0/1145237020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078530 msgr2=0x7fc3a4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 --2- 172.21.15.12:0/1145237020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078530 0x7fc3a4078990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc390009900 tx=0x7fc39002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 -- 172.21.15.12:0/1145237020 shutdown_connections 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 --2- 172.21.15.12:0/1145237020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078ed0 0x7fc3a4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 --2- 172.21.15.12:0/1145237020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078530 0x7fc3a4078990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 --2- 172.21.15.12:0/1145237020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 0x7fc3a4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 -- 172.21.15.12:0/1145237020 >> 172.21.15.12:0/1145237020 conn(0x7fc3a41000e0 msgr2=0x7fc3a4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 -- 172.21.15.12:0/1145237020 shutdown_connections 2024-04-26T22:02:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.634+0000 7fc3aae00640 1 -- 172.21.15.12:0/1145237020 wait complete. 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.635+0000 7fc3aae00640 1 Processor -- start 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 -- start start 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 0x7fc3a419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 0x7fc3a41a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4114b10 con 0x7fc3a4078ed0 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4114990 con 0x7fc3a40772f0 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3aae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4114c90 con 0x7fc3a4078530 2024-04-26T22:02:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.636+0000 7fc3a8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 0x7fc3a41a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56558/0 (socket says 172.21.15.12:56558) 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a3400640 1 -- 172.21.15.12:0/327920506 learned_addr learned my addr 172.21.15.12:0/327920506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a3e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 0x7fc3a419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 -- 172.21.15.12:0/327920506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 msgr2=0x7fc3a419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 -- 172.21.15.12:0/327920506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 msgr2=0x7fc3a419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 0x7fc3a419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 -- 172.21.15.12:0/327920506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a41a6dc0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a3400640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.637+0000 7fc3a8e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 0x7fc3a41a6750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009fd0 tx=0x7fc39c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.638+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00ce20 con 0x7fc3a4078ed0 2024-04-26T22:02:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.638+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc39c005020 con 0x7fc3a4078ed0 2024-04-26T22:02:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.638+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a41a70b0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.638+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c010690 con 0x7fc3a4078ed0 2024-04-26T22:02:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.638+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a41a7600 con 0x7fc3a4078ed0 2024-04-26T22:02:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.639+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a4111690 con 0x7fc3a4078ed0 2024-04-26T22:02:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.641+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc39c0054d0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.643+0000 7fc3a0c00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 0x7fc3880a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.643+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc39c0d0ec0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.643+0000 7fc3a3e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 0x7fc3880a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.644+0000 7fc3a3e00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 0x7fc3880a6250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc394009bd0 tx=0x7fc394005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:19.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.646+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc39c09d3c0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.855+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3a4063530 con 0x7fc3a4078ed0 2024-04-26T22:02:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.856+0000 7fc3a0c00640 1 -- 172.21.15.12:0/327920506 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fc39c09ccc0 con 0x7fc3a4078ed0 2024-04-26T22:02:19.857 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 msgr2=0x7fc3880a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 0x7fc3880a6250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc394009bd0 tx=0x7fc394005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 msgr2=0x7fc3a41a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 0x7fc3a41a6750 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009fd0 tx=0x7fc39c00ef10 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 shutdown_connections 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc3880a3d90 0x7fc3880a6250 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3a4078ed0 0x7fc3a41a6750 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3a4078530 0x7fc3a419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 --2- 172.21.15.12:0/327920506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3a40772f0 0x7fc3a419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 >> 172.21.15.12:0/327920506 conn(0x7fc3a41000e0 msgr2=0x7fc3a41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 shutdown_connections 2024-04-26T22:02:19.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:19.859+0000 7fc3aae00640 1 -- 172.21.15.12:0/327920506 wait complete. 2024-04-26T22:02:19.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:02:20.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 -- 172.21.15.12:0/2109703265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f579410a180 msgr2=0x7f579410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 --2- 172.21.15.12:0/2109703265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f579410a180 0x7f579410a600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 -- 172.21.15.12:0/2109703265 shutdown_connections 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 --2- 172.21.15.12:0/2109703265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f57941113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 --2- 172.21.15.12:0/2109703265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f579410a180 0x7f579410a600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 --2- 172.21.15.12:0/2109703265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5794108f90 0x7f57941093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 -- 172.21.15.12:0/2109703265 >> 172.21.15.12:0/2109703265 conn(0x7f5794077f30 msgr2=0x7f579407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 -- 172.21.15.12:0/2109703265 shutdown_connections 2024-04-26T22:02:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.086+0000 7f5799a00640 1 -- 172.21.15.12:0/2109703265 wait complete. 2024-04-26T22:02:20.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.087+0000 7f5799a00640 1 Processor -- start 2024-04-26T22:02:20.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.087+0000 7f5799a00640 1 -- start start 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 0x7f579419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f579419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 0x7f57941a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5794114b20 con 0x7f5794108f90 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f57941149a0 con 0x7f579410ab40 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5799a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5794114ca0 con 0x7f57941a4990 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5792a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 0x7f579419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5792a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 0x7f579419f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34354/0 (socket says 172.21.15.12:34354) 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5792000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f579419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5792a00640 1 -- 172.21.15.12:0/3715544736 learned_addr learned my addr 172.21.15.12:0/3715544736 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.088+0000 7f5793400640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 0x7f57941a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 -- 172.21.15.12:0/3715544736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 msgr2=0x7f57941a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 0x7f57941a6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 -- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 msgr2=0x7f579419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 0x7f579419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 -- 172.21.15.12:0/3715544736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57941a73f0 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5793400640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 0x7f57941a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5792000640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f579419f930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5780008000 tx=0x7f5780003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578003f020 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.089+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57941a7680 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.090+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5780031c50 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.090+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780035070 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.090+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941a7b40 con 0x7f579410ab40 2024-04-26T22:02:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.091+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57941103c0 con 0x7f579410ab40 2024-04-26T22:02:20.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.093+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5780042020 con 0x7f579410ab40 2024-04-26T22:02:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.094+0000 7f578b400640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 0x7f57780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.094+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f57800f1b60 con 0x7f579410ab40 2024-04-26T22:02:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.095+0000 7f5792a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 0x7f57780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.095+0000 7f5792a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 0x7f57780a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57840058c0 tx=0x7f5784005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:20.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.097+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5780049020 con 0x7f579410ab40 2024-04-26T22:02:20.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.304+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f579402dc60 con 0x7f579410ab40 2024-04-26T22:02:20.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.305+0000 7f578b400640 1 -- 172.21.15.12:0/3715544736 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f57800bdfe0 con 0x7f579410ab40 2024-04-26T22:02:20.306 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:02:20.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 msgr2=0x7f57780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 0x7f57780a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57840058c0 tx=0x7f5784005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:20.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 msgr2=0x7f579419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f579419f930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5780008000 tx=0x7f5780003c10 comp rx=0 tx=0).stop 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 shutdown_connections 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f57780a3cc0 0x7f57780a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57941a4990 0x7f57941a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579410ab40 0x7f579419f930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 --2- 172.21.15.12:0/3715544736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5794108f90 0x7f579419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 >> 172.21.15.12:0/3715544736 conn(0x7f5794077f30 msgr2=0x7f579410d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 shutdown_connections 2024-04-26T22:02:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.308+0000 7f5799a00640 1 -- 172.21.15.12:0/3715544736 wait complete. 2024-04-26T22:02:20.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 -- 172.21.15.12:0/1271611369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c10ed20 msgr2=0x7fef9c111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 --2- 172.21.15.12:0/1271611369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c10ed20 0x7fef9c111110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef94009cd0 tx=0x7fef9402a8f0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 -- 172.21.15.12:0/1271611369 shutdown_connections 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 --2- 172.21.15.12:0/1271611369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c10ed20 0x7fef9c111110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 --2- 172.21.15.12:0/1271611369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c075390 0x7fef9c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 --2- 172.21.15.12:0/1271611369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c076970 0x7fef9c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 -- 172.21.15.12:0/1271611369 >> 172.21.15.12:0/1271611369 conn(0x7fef9c0fdcc0 msgr2=0x7fef9c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:20.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 -- 172.21.15.12:0/1271611369 shutdown_connections 2024-04-26T22:02:20.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.532+0000 7fefa3000640 1 -- 172.21.15.12:0/1271611369 wait complete. 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 Processor -- start 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 -- start start 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c076970 0x7fef9c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 0x7fef9c1a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1148c0 con 0x7fef9c075390 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c114740 con 0x7fef9c10ed20 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.533+0000 7fefa3000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c114a40 con 0x7fef9c076970 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34364/0 (socket says 172.21.15.12:34364) 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 -- 172.21.15.12:0/767173713 learned_addr learned my addr 172.21.15.12:0/767173713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa2600640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 0x7fef9c1a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1200640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c076970 0x7fef9c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 -- 172.21.15.12:0/767173713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c076970 msgr2=0x7fef9c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c076970 0x7fef9c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 -- 172.21.15.12:0/767173713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 msgr2=0x7fef9c1a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 0x7fef9c1a6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa1c00640 1 -- 172.21.15.12:0/767173713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a6b90 con 0x7fef9c075390 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.534+0000 7fefa2600640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 0x7fef9c1a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fefa1c00640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9f0 tx=0x7fef9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000cd80 con 0x7fef9c075390 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef900108e0 con 0x7fef9c075390 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a6e80 con 0x7fef9c075390 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90018980 con 0x7fef9c075390 2024-04-26T22:02:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.535+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a72e0 con 0x7fef9c075390 2024-04-26T22:02:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.536+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c104ef0 con 0x7fef9c075390 2024-04-26T22:02:20.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.538+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef90010430 con 0x7fef9c075390 2024-04-26T22:02:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.541+0000 7fef9a000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.541+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fef900d0c60 con 0x7fef9c075390 2024-04-26T22:02:20.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.541+0000 7fefa1200640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.542+0000 7fefa1200640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 0x7fef840a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef9c1a0480 tx=0x7fef88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.545+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fef9009ca90 con 0x7fef9c075390 2024-04-26T22:02:20.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:20 smithi012 ceph-mon[28184]: pgmap v402: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:20.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:20 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4019119713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:20.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:20 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/327920506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:20 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3715544736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:20 smithi154 ceph-mon[32578]: pgmap v402: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:20.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:20 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4019119713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:20.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:20 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/327920506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:20 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3715544736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:20 smithi077 ceph-mon[32280]: pgmap v402: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:20.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:20 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4019119713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:20.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:20 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/327920506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:20 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3715544736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:20.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.752+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef9c063630 con 0x7fef9c075390 2024-04-26T22:02:20.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.753+0000 7fef9a000640 1 -- 172.21.15.12:0/767173713 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fef900a0d90 con 0x7fef9c075390 2024-04-26T22:02:20.754 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:02:20.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 msgr2=0x7fef840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 0x7fef840a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef9c1a0480 tx=0x7fef88005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 msgr2=0x7fef9c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9f0 tx=0x7fef9000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 shutdown_connections 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef9c10ed20 0x7fef9c1a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef9c076970 0x7fef9c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 --2- 172.21.15.12:0/767173713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef9c075390 0x7fef9c19ef70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.755+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 >> 172.21.15.12:0/767173713 conn(0x7fef9c0fdcc0 msgr2=0x7fef9c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.756+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 shutdown_connections 2024-04-26T22:02:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.756+0000 7fefa3000640 1 -- 172.21.15.12:0/767173713 wait complete. 2024-04-26T22:02:20.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.994+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1999473656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 msgr2=0x7f8b34104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.994+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1999473656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b34104fd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009ca0 tx=0x7f8b2402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1999473656 shutdown_connections 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1999473656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 0x7f8b3410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1999473656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b34106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1999473656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b34104fd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1999473656 >> 172.21.15.12:0/1999473656 conn(0x7f8b34100120 msgr2=0x7f8b34102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1999473656 shutdown_connections 2024-04-26T22:02:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.995+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1999473656 wait complete. 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 Processor -- start 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 -- start start 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b3419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 0x7f8b341a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34110700 con 0x7f8b34105da0 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34110580 con 0x7f8b34106760 2024-04-26T22:02:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.996+0000 7f8b3ac00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34110880 con 0x7f8b34104bb0 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34378/0 (socket says 172.21.15.12:34378) 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 -- 172.21.15.12:0/1660270669 learned_addr learned my addr 172.21.15.12:0/1660270669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b38c00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 0x7f8b341a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33e00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b3419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 -- 172.21.15.12:0/1660270669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 msgr2=0x7f8b3419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b3419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 -- 172.21.15.12:0/1660270669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 msgr2=0x7f8b341a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 0x7f8b341a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 -- 172.21.15.12:0/1660270669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341a2a20 con 0x7f8b34105da0 2024-04-26T22:02:21.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.997+0000 7f8b33400640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e9f0 tx=0x7f8b1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.998+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00cd80 con 0x7f8b34105da0 2024-04-26T22:02:21.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.998+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1c0108e0 con 0x7f8b34105da0 2024-04-26T22:02:21.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.998+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c018980 con 0x7f8b34105da0 2024-04-26T22:02:21.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.998+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b341a2d10 con 0x7f8b34105da0 2024-04-26T22:02:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.998+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b341a3200 con 0x7f8b34105da0 2024-04-26T22:02:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:20.999+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3410bf50 con 0x7f8b34105da0 2024-04-26T22:02:21.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.001+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b1c010430 con 0x7f8b34105da0 2024-04-26T22:02:21.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.003+0000 7f8b30c00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 0x7f8b180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.003+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8b1c0d0c40 con 0x7f8b34105da0 2024-04-26T22:02:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.003+0000 7f8b33e00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 0x7f8b180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.004+0000 7f8b33e00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 0x7f8b180a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000f80 tx=0x7f8b24005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.007+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b1c09ca70 con 0x7f8b34105da0 2024-04-26T22:02:21.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.210+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8b3402dd20 con 0x7f8b34105da0 2024-04-26T22:02:21.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.211+0000 7f8b30c00640 1 -- 172.21.15.12:0/1660270669 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f8b1c0a0d70 con 0x7f8b34105da0 2024-04-26T22:02:21.212 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:02:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.213+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 msgr2=0x7f8b180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.213+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 0x7f8b180a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000f80 tx=0x7f8b24005f30 comp rx=0 tx=0).stop 2024-04-26T22:02:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.213+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 msgr2=0x7f8b3419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.213+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e9f0 tx=0x7f8b1c00eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 shutdown_connections 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f8b180a3c40 0x7f8b180a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b34106760 0x7f8b341a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b34105da0 0x7f8b3419b340 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 --2- 172.21.15.12:0/1660270669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b34104bb0 0x7f8b3419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 >> 172.21.15.12:0/1660270669 conn(0x7f8b34100120 msgr2=0x7f8b34109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 shutdown_connections 2024-04-26T22:02:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.214+0000 7f8b3ac00640 1 -- 172.21.15.12:0/1660270669 wait complete. 2024-04-26T22:02:21.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 -- 172.21.15.12:0/3525194608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464106740 msgr2=0x7fc46410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 --2- 172.21.15.12:0/3525194608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464106740 0x7fc46410cfd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009be0 tx=0x7fc45c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 -- 172.21.15.12:0/3525194608 shutdown_connections 2024-04-26T22:02:21.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 --2- 172.21.15.12:0/3525194608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464106740 0x7fc46410cfd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 --2- 172.21.15.12:0/3525194608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464105d80 0x7fc464106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 --2- 172.21.15.12:0/3525194608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc464104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 -- 172.21.15.12:0/3525194608 >> 172.21.15.12:0/3525194608 conn(0x7fc464100120 msgr2=0x7fc464102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 -- 172.21.15.12:0/3525194608 shutdown_connections 2024-04-26T22:02:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.430+0000 7fc46c600640 1 -- 172.21.15.12:0/3525194608 wait complete. 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.431+0000 7fc46c600640 1 Processor -- start 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 -- start start 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464105d80 0x7fc46419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464106740 0x7fc4641a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641106a0 con 0x7fc464104b90 2024-04-26T22:02:21.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc464110520 con 0x7fc464106740 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc46c600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc464110820 con 0x7fc464105d80 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc469c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc469c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34404/0 (socket says 172.21.15.12:34404) 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.432+0000 7fc469c00640 1 -- 172.21.15.12:0/1243082272 learned_addr learned my addr 172.21.15.12:0/1243082272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469200640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464105d80 0x7fc46419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc46a600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464106740 0x7fc4641a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 -- 172.21.15.12:0/1243082272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464105d80 msgr2=0x7fc46419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464105d80 0x7fc46419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 -- 172.21.15.12:0/1243082272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464106740 msgr2=0x7fc4641a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464106740 0x7fc4641a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 -- 172.21.15.12:0/1243082272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4641a2940 con 0x7fc464104b90 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.433+0000 7fc469c00640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9f0 tx=0x7fc45800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.434+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ccc0 con 0x7fc464104b90 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.434+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4580035f0 con 0x7fc464104b90 2024-04-26T22:02:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.434+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4641a2c30 con 0x7fc464104b90 2024-04-26T22:02:21.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.434+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ce60 con 0x7fc464104b90 2024-04-26T22:02:21.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.434+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a31d0 con 0x7fc464104b90 2024-04-26T22:02:21.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.435+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46410bf00 con 0x7fc464104b90 2024-04-26T22:02:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.437+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc458003fb0 con 0x7fc464104b90 2024-04-26T22:02:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.439+0000 7fc462000640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 0x7fc44c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.439+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc4580d0b90 con 0x7fc464104b90 2024-04-26T22:02:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.439+0000 7fc469200640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 0x7fc44c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.440+0000 7fc469200640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 0x7fc44c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc450009bd0 tx=0x7fc450005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.443+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc45809d010 con 0x7fc464104b90 2024-04-26T22:02:21.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:21 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/767173713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:21 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1660270669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:21 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/767173713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:21 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1660270669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.691+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46402dd80 con 0x7fc464104b90 2024-04-26T22:02:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.692+0000 7fc462000640 1 -- 172.21.15.12:0/1243082272 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7fc45809c910 con 0x7fc464104b90 2024-04-26T22:02:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:21.693 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:02:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.694+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 msgr2=0x7fc44c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 0x7fc44c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc450009bd0 tx=0x7fc450005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 msgr2=0x7fc46419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9f0 tx=0x7fc45800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 shutdown_connections 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fc44c0a3cc0 0x7fc44c0a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc464106740 0x7fc4641a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc464105d80 0x7fc46419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 --2- 172.21.15.12:0/1243082272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc464104b90 0x7fc46419ad30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 >> 172.21.15.12:0/1243082272 conn(0x7fc464100120 msgr2=0x7fc4641090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 shutdown_connections 2024-04-26T22:02:21.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.695+0000 7fc46c600640 1 -- 172.21.15.12:0/1243082272 wait complete. 2024-04-26T22:02:21.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:02:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:21 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/767173713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:21 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1660270669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 -- 172.21.15.12:0/3765221302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678105b70 msgr2=0x7fd678105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 --2- 172.21.15.12:0/3765221302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678105b70 0x7fd678105ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd660009860 tx=0x7fd66002a340 comp rx=0 tx=0).stop 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 -- 172.21.15.12:0/3765221302 shutdown_connections 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 --2- 172.21.15.12:0/3765221302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678106530 0x7fd67810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 --2- 172.21.15.12:0/3765221302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678105b70 0x7fd678105ff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 --2- 172.21.15.12:0/3765221302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678104980 0x7fd678104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 -- 172.21.15.12:0/3765221302 >> 172.21.15.12:0/3765221302 conn(0x7fd6780fff10 msgr2=0x7fd678102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 -- 172.21.15.12:0/3765221302 shutdown_connections 2024-04-26T22:02:21.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.932+0000 7fd677e00640 1 -- 172.21.15.12:0/3765221302 wait complete. 2024-04-26T22:02:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.933+0000 7fd677e00640 1 Processor -- start 2024-04-26T22:02:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.933+0000 7fd677e00640 1 -- start start 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 0x7fd67819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678106530 0x7fd6781a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781104b0 con 0x7fd678105b70 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110330 con 0x7fd678104980 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd678110630 con 0x7fd678106530 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34412/0 (socket says 172.21.15.12:34412) 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd66fe00640 1 -- 172.21.15.12:0/3124240805 learned_addr learned my addr 172.21.15.12:0/3124240805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd676a00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 0x7fd67819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.934+0000 7fd677400640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678106530 0x7fd6781a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 -- 172.21.15.12:0/3124240805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678106530 msgr2=0x7fd6781a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678106530 0x7fd6781a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 -- 172.21.15.12:0/3124240805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 msgr2=0x7fd67819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 0x7fd67819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 -- 172.21.15.12:0/3124240805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6781a2780 con 0x7fd678105b70 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd676a00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 0x7fd67819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66fe00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6600057c0 tx=0x7fd660032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660037040 con 0x7fd678105b70 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd660030bd0 con 0x7fd678105b70 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6781a2a10 con 0x7fd678105b70 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.935+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660005280 con 0x7fd678105b70 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.936+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a2ea0 con 0x7fd678105b70 2024-04-26T22:02:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.937+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67810bca0 con 0x7fd678105b70 2024-04-26T22:02:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.939+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd660043020 con 0x7fd678105b70 2024-04-26T22:02:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.941+0000 7fd66f400640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 0x7fd65c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.942+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd6600efe70 con 0x7fd678105b70 2024-04-26T22:02:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.942+0000 7fd676a00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 0x7fd65c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.943+0000 7fd676a00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 0x7fd65c0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6680096c0 tx=0x7fd668009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:21.947+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6600bc460 con 0x7fd678105b70 2024-04-26T22:02:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.154+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd67802dce0 con 0x7fd678105b70 2024-04-26T22:02:22.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.159+0000 7fd66f400640 1 -- 172.21.15.12:0/3124240805 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7fd6600bbd60 con 0x7fd678105b70 2024-04-26T22:02:22.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:22.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.162+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 msgr2=0x7fd65c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.162+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 0x7fd65c0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6680096c0 tx=0x7fd668009260 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.162+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 msgr2=0x7fd67819b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.162+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6600057c0 tx=0x7fd660032da0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 shutdown_connections 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd65c0a3c40 0x7fd65c0a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd678106530 0x7fd6781a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd678105b70 0x7fd67819b0a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 --2- 172.21.15.12:0/3124240805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd678104980 0x7fd67819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 >> 172.21.15.12:0/3124240805 conn(0x7fd6780fff10 msgr2=0x7fd678108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 shutdown_connections 2024-04-26T22:02:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.163+0000 7fd677e00640 1 -- 172.21.15.12:0/3124240805 wait complete. 2024-04-26T22:02:22.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:02:22.183 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:02:22.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.402+0000 7f7df4a00640 1 -- 172.21.15.12:0/1062156469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0102840 msgr2=0x7f7df0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.402+0000 7f7df4a00640 1 --2- 172.21.15.12:0/1062156469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0102840 0x7f7df0102c60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009860 tx=0x7f7de002a340 comp rx=0 tx=0).stop 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 -- 172.21.15.12:0/1062156469 shutdown_connections 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 --2- 172.21.15.12:0/1062156469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df01043f0 0x7f7df010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 --2- 172.21.15.12:0/1062156469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df0103a30 0x7f7df0103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 --2- 172.21.15.12:0/1062156469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0102840 0x7f7df0102c60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 -- 172.21.15.12:0/1062156469 >> 172.21.15.12:0/1062156469 conn(0x7f7df00fddd0 msgr2=0x7f7df01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 -- 172.21.15.12:0/1062156469 shutdown_connections 2024-04-26T22:02:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.403+0000 7f7df4a00640 1 -- 172.21.15.12:0/1062156469 wait complete. 2024-04-26T22:02:22.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.404+0000 7f7df4a00640 1 Processor -- start 2024-04-26T22:02:22.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.404+0000 7f7df4a00640 1 -- start start 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 0x7f7df019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df01043f0 0x7f7df01a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010e370 con 0x7f7df01043f0 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010e1f0 con 0x7f7df0102840 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7df4a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010e4f0 con 0x7f7df0103a30 2024-04-26T22:02:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7dee800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7dee800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52462/0 (socket says 172.21.15.12:52462) 2024-04-26T22:02:22.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7dee800640 1 -- 172.21.15.12:0/911028282 learned_addr learned my addr 172.21.15.12:0/911028282 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:22.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7def200640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df01043f0 0x7f7df01a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.405+0000 7f7dede00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 0x7f7df019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 -- 172.21.15.12:0/911028282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 msgr2=0x7f7df019b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 0x7f7df019b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 -- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df01043f0 msgr2=0x7f7df01a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df01043f0 0x7f7df01a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 -- 172.21.15.12:0/911028282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df01a2890 con 0x7f7df0102840 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dede00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 0x7f7df019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7dee800640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7de00057c0 tx=0x7f7de0032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.406+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0037040 con 0x7f7df0102840 2024-04-26T22:02:22.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.407+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01a2b20 con 0x7f7df0102840 2024-04-26T22:02:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.407+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de0030bd0 con 0x7f7df0102840 2024-04-26T22:02:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.407+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0005280 con 0x7f7df0102840 2024-04-26T22:02:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.407+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01a3000 con 0x7f7df0102840 2024-04-26T22:02:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.408+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df01a31d0 con 0x7f7df0102840 2024-04-26T22:02:22.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.410+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7de0043020 con 0x7f7df0102840 2024-04-26T22:02:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.411+0000 7f7de7400640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 0x7f7dd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.411+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7de00efdf0 con 0x7f7df0102840 2024-04-26T22:02:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.412+0000 7f7dede00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 0x7f7dd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.412+0000 7f7dede00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 0x7f7dd40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80096c0 tx=0x7f7dd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:22.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.416+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7de00bc380 con 0x7f7df0102840 2024-04-26T22:02:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:22 smithi012 ceph-mon[28184]: pgmap v403: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:22 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1243082272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:22 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3124240805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.623+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7df01a33a0 con 0x7f7df0102840 2024-04-26T22:02:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.624+0000 7f7de7400640 1 -- 172.21.15.12:0/911028282 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f7de00bbc80 con 0x7f7df0102840 2024-04-26T22:02:22.625 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 msgr2=0x7f7dd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 0x7f7dd40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80096c0 tx=0x7f7dd8009260 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 msgr2=0x7f7df019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7de00057c0 tx=0x7f7de0032da0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 shutdown_connections 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f7dd40a3cc0 0x7f7dd40a6180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7df01043f0 0x7f7df01a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7df0103a30 0x7f7df019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 --2- 172.21.15.12:0/911028282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7df0102840 0x7f7df019ac80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 >> 172.21.15.12:0/911028282 conn(0x7f7df00fddd0 msgr2=0x7f7df0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 shutdown_connections 2024-04-26T22:02:22.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.627+0000 7f7df4a00640 1 -- 172.21.15.12:0/911028282 wait complete. 2024-04-26T22:02:22.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:22 smithi154 ceph-mon[32578]: pgmap v403: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:22.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:22 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1243082272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:22 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3124240805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:02:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:22 smithi077 ceph-mon[32280]: pgmap v403: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:22 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1243082272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:22 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3124240805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 -- 172.21.15.12:0/3709819150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17010ef40 msgr2=0x7fe170111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 --2- 172.21.15.12:0/3709819150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17010ef40 0x7fe170111330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe168009ca0 tx=0x7fe16802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 -- 172.21.15.12:0/3709819150 shutdown_connections 2024-04-26T22:02:22.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 --2- 172.21.15.12:0/3709819150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17010ef40 0x7fe170111330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 --2- 172.21.15.12:0/3709819150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe170075390 0x7fe170075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 --2- 172.21.15.12:0/3709819150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe170074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 -- 172.21.15.12:0/3709819150 >> 172.21.15.12:0/3709819150 conn(0x7fe1700fdee0 msgr2=0x7fe170100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 -- 172.21.15.12:0/3709819150 shutdown_connections 2024-04-26T22:02:22.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.867+0000 7fe176600640 1 -- 172.21.15.12:0/3709819150 wait complete. 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.868+0000 7fe176600640 1 Processor -- start 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.868+0000 7fe176600640 1 -- start start 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe17019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 0x7fe1701a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114ae0 con 0x7fe17010ef40 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114960 con 0x7fe170076970 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe176600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114c60 con 0x7fe170075390 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 0x7fe1701a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 0x7fe1701a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34454/0 (socket says 172.21.15.12:34454) 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56664/0 (socket says 172.21.15.12:56664) 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16fe00640 1 -- 172.21.15.12:0/3397585249 learned_addr learned my addr 172.21.15.12:0/3397585249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.869+0000 7fe16ea00640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe17019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 -- 172.21.15.12:0/3397585249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 msgr2=0x7fe17019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe17019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 -- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 msgr2=0x7fe1701a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 0x7fe1701a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 -- 172.21.15.12:0/3397585249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a6d80 con 0x7fe170075390 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16ea00640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe17019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe16f400640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16000ccc0 con 0x7fe170075390 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe160005020 con 0x7fe170075390 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a7070 con 0x7fe170075390 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160004a00 con 0x7fe170075390 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.870+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a75c0 con 0x7fe170075390 2024-04-26T22:02:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.871+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170105000 con 0x7fe170075390 2024-04-26T22:02:22.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.874+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe160004ba0 con 0x7fe170075390 2024-04-26T22:02:22.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.875+0000 7fe167e00640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 0x7fe1540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:22.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.875+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe1600d0890 con 0x7fe170075390 2024-04-26T22:02:22.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.875+0000 7fe16ea00640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 0x7fe1540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:22.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.876+0000 7fe16ea00640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 0x7fe1540a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1701a0670 tx=0x7fe15c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:22.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:22.877+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe16009cda0 con 0x7fe170075390 2024-04-26T22:02:23.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.087+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe170063530 con 0x7fe170075390 2024-04-26T22:02:23.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.088+0000 7fe167e00640 1 -- 172.21.15.12:0/3397585249 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fe16009c6a0 con 0x7fe170075390 2024-04-26T22:02:23.089 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:02:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 msgr2=0x7fe1540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 0x7fe1540a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1701a0670 tx=0x7fe15c00b410 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 msgr2=0x7fe17019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 shutdown_connections 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fe1540a3cc0 0x7fe1540a6180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe17010ef40 0x7fe1701a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe170076970 0x7fe17019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 --2- 172.21.15.12:0/3397585249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe170075390 0x7fe17019f160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.090+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 >> 172.21.15.12:0/3397585249 conn(0x7fe1700fdee0 msgr2=0x7fe1701021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.091+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 shutdown_connections 2024-04-26T22:02:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.091+0000 7fe176600640 1 -- 172.21.15.12:0/3397585249 wait complete. 2024-04-26T22:02:23.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:02:23.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.326+0000 7f760ac00640 1 -- 172.21.15.12:0/3156247301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76041033f0 msgr2=0x7f76041057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.326+0000 7f760ac00640 1 --2- 172.21.15.12:0/3156247301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76041033f0 0x7f76041057e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009900 tx=0x7f75ec02a410 comp rx=0 tx=0).stop 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 -- 172.21.15.12:0/3156247301 shutdown_connections 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 --2- 172.21.15.12:0/3156247301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7604108650 0x7f760410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 --2- 172.21.15.12:0/3156247301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f7604108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 --2- 172.21.15.12:0/3156247301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76041033f0 0x7f76041057e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 -- 172.21.15.12:0/3156247301 >> 172.21.15.12:0/3156247301 conn(0x7f76040fcde0 msgr2=0x7f76040ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 -- 172.21.15.12:0/3156247301 shutdown_connections 2024-04-26T22:02:23.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.327+0000 7f760ac00640 1 -- 172.21.15.12:0/3156247301 wait complete. 2024-04-26T22:02:23.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 Processor -- start 2024-04-26T22:02:23.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 -- start start 2024-04-26T22:02:23.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76041033f0 0x7f760419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 0x7f76041a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f760410e510 con 0x7f7604108650 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f760410e390 con 0x7f7604105d20 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.328+0000 7f760ac00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f760410e690 con 0x7f76041033f0 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7608c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 0x7f76041a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76041033f0 0x7f760419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52490/0 (socket says 172.21.15.12:52490) 2024-04-26T22:02:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 -- 172.21.15.12:0/3815837359 learned_addr learned my addr 172.21.15.12:0/3815837359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:23.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 -- 172.21.15.12:0/3815837359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76041033f0 msgr2=0x7f760419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76041033f0 0x7f760419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 -- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 msgr2=0x7f76041a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 0x7f76041a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 -- 172.21.15.12:0/3815837359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76041a29e0 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7608c00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 0x7f76041a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.329+0000 7f7603400640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75f400eb90 tx=0x7f75f400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.330+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4010630 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.330+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f400ee90 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.330+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76041a2cd0 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.330+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4018ac0 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.330+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76041a3210 con 0x7f7604105d20 2024-04-26T22:02:23.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.331+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760410a1a0 con 0x7f7604105d20 2024-04-26T22:02:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.333+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f75f4018c60 con 0x7f7604105d20 2024-04-26T22:02:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.335+0000 7f7600c00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 0x7f75e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.335+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f75f40d0ec0 con 0x7f7604105d20 2024-04-26T22:02:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.335+0000 7f7603e00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 0x7f75e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.336+0000 7f7603e00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 0x7f75e80a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75ec031500 tx=0x7f75ec005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:23.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.339+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f75f409ccf0 con 0x7f7604105d20 2024-04-26T22:02:23.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.545+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f760402dd20 con 0x7f7604105d20 2024-04-26T22:02:23.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.546+0000 7f7600c00640 1 -- 172.21.15.12:0/3815837359 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f75f40a0ff0 con 0x7f7604105d20 2024-04-26T22:02:23.547 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:02:23.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 msgr2=0x7f75e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 0x7f75e80a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75ec031500 tx=0x7f75ec005a20 comp rx=0 tx=0).stop 2024-04-26T22:02:23.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 msgr2=0x7f760419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75f400eb90 tx=0x7f75f400c6a0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 shutdown_connections 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f75e80a3cc0 0x7f75e80a6180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7604108650 0x7f76041a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7604105d20 0x7f760419b300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 --2- 172.21.15.12:0/3815837359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76041033f0 0x7f760419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 >> 172.21.15.12:0/3815837359 conn(0x7f76040fcde0 msgr2=0x7f7604102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 shutdown_connections 2024-04-26T22:02:23.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.549+0000 7f760ac00640 1 -- 172.21.15.12:0/3815837359 wait complete. 2024-04-26T22:02:23.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/911028282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.567 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3397585249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:02:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/911028282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3397585249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/911028282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3397585249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 -- 172.21.15.12:0/260258139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 msgr2=0x7f7838075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 --2- 172.21.15.12:0/260258139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f7838075810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7824009900 tx=0x7f782402a400 comp rx=0 tx=0).stop 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 -- 172.21.15.12:0/260258139 shutdown_connections 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 --2- 172.21.15.12:0/260258139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f783810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 --2- 172.21.15.12:0/260258139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f7838075810 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 --2- 172.21.15.12:0/260258139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 0x7f7838074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 -- 172.21.15.12:0/260258139 >> 172.21.15.12:0/260258139 conn(0x7f78380fddd0 msgr2=0x7f78381001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 -- 172.21.15.12:0/260258139 shutdown_connections 2024-04-26T22:02:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.787+0000 7f783f000640 1 -- 172.21.15.12:0/260258139 wait complete. 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.788+0000 7f783f000640 1 Processor -- start 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.788+0000 7f783f000640 1 -- start start 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 0x7f783819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f78381a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7838110530 con 0x7f7838075390 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78381103b0 con 0x7f783810a700 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783f000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78381106b0 con 0x7f7838076970 2024-04-26T22:02:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34480/0 (socket says 172.21.15.12:34480) 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 -- 172.21.15.12:0/2517982232 learned_addr learned my addr 172.21.15.12:0/2517982232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783d200640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 0x7f783819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783e600640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f78381a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 -- 172.21.15.12:0/2517982232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 msgr2=0x7f783819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 0x7f783819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 -- 172.21.15.12:0/2517982232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 msgr2=0x7f78381a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f78381a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783dc00640 1 -- 172.21.15.12:0/2517982232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78381a2890 con 0x7f7838075390 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.789+0000 7f783e600640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f78381a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f783dc00640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f782c00d950 tx=0x7f782c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c017070 con 0x7f7838075390 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782c009c30 con 0x7f7838075390 2024-04-26T22:02:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c00f620 con 0x7f7838075390 2024-04-26T22:02:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78381a2b80 con 0x7f7838075390 2024-04-26T22:02:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.790+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381a3120 con 0x7f7838075390 2024-04-26T22:02:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.791+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838109430 con 0x7f7838075390 2024-04-26T22:02:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.793+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f782c010460 con 0x7f7838075390 2024-04-26T22:02:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.795+0000 7f7836000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 0x7f78200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.795+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f782c0d05f0 con 0x7f7838075390 2024-04-26T22:02:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.795+0000 7f783d200640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 0x7f78200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.796+0000 7f783d200640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 0x7f78200a6180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7824000c00 tx=0x7f78240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:23.799+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f782c09ca70 con 0x7f7838075390 2024-04-26T22:02:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.007+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f783802dd40 con 0x7f7838075390 2024-04-26T22:02:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.008+0000 7f7836000640 1 -- 172.21.15.12:0/2517982232 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f782c009de0 con 0x7f7838075390 2024-04-26T22:02:24.009 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 msgr2=0x7f78200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 0x7f78200a6180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7824000c00 tx=0x7f78240314a0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 msgr2=0x7f783819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f782c00d950 tx=0x7f782c00de10 comp rx=0 tx=0).stop 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 shutdown_connections 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f78200a3cc0 0x7f78200a6180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f783810a700 0x7f78381a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7838076970 0x7f783819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 --2- 172.21.15.12:0/2517982232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7838075390 0x7f783819ac80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 >> 172.21.15.12:0/2517982232 conn(0x7f78380fddd0 msgr2=0x7f7838106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 shutdown_connections 2024-04-26T22:02:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.011+0000 7f783f000640 1 -- 172.21.15.12:0/2517982232 wait complete. 2024-04-26T22:02:24.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:24.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 -- 172.21.15.12:0/451899236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b34106db0 msgr2=0x7f3b341091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 --2- 172.21.15.12:0/451899236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b34106db0 0x7f3b341091a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009cd0 tx=0x7f3b2c02a8f0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 -- 172.21.15.12:0/451899236 shutdown_connections 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 --2- 172.21.15.12:0/451899236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b34106db0 0x7f3b341091a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 --2- 172.21.15.12:0/451899236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b341020b0 0x7f3b34102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 --2- 172.21.15.12:0/451899236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 0x7f3b34101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 -- 172.21.15.12:0/451899236 >> 172.21.15.12:0/451899236 conn(0x7f3b340fbef0 msgr2=0x7f3b340fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 -- 172.21.15.12:0/451899236 shutdown_connections 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.246+0000 7f3b33e00640 1 -- 172.21.15.12:0/451899236 wait complete. 2024-04-26T22:02:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 Processor -- start 2024-04-26T22:02:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 -- start start 2024-04-26T22:02:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 0x7f3b3419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 0x7f3b341a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410c0e0 con 0x7f3b341020b0 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410bf60 con 0x7f3b34106db0 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.248+0000 7f3b33e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410c260 con 0x7f3b34068de0 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b33400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 0x7f3b341a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34492/0 (socket says 172.21.15.12:34492) 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 -- 172.21.15.12:0/1345217696 learned_addr learned my addr 172.21.15.12:0/1345217696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b33400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 0x7f3b341a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52516/0 (socket says 172.21.15.12:52516) 2024-04-26T22:02:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32a00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 0x7f3b3419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 -- 172.21.15.12:0/1345217696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 msgr2=0x7f3b3419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 0x7f3b3419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 -- 172.21.15.12:0/1345217696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 msgr2=0x7f3b341a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 0x7f3b341a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.249+0000 7f3b32000640 1 -- 172.21.15.12:0/1345217696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b341a27b0 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b32000640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00e9c0 tx=0x7f3b1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00cd80 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b1c0035f0 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c010690 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341a2aa0 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.250+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a2fa0 con 0x7f3b341020b0 2024-04-26T22:02:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.252+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b34105bc0 con 0x7f3b341020b0 2024-04-26T22:02:24.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.254+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b1c0037a0 con 0x7f3b341020b0 2024-04-26T22:02:24.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.255+0000 7f3b2b400640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 0x7f3b180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.256+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3b1c0d0c90 con 0x7f3b341020b0 2024-04-26T22:02:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.256+0000 7f3b32a00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 0x7f3b180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.257+0000 7f3b32a00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 0x7f3b180a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3b341019d0 tx=0x7f3b24010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:24.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.260+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3b1c09d110 con 0x7f3b341020b0 2024-04-26T22:02:24.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.510+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b3402dad0 con 0x7f3b341020b0 2024-04-26T22:02:24.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.512+0000 7f3b2b400640 1 -- 172.21.15.12:0/1345217696 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7f3b1c09ca10 con 0x7f3b341020b0 2024-04-26T22:02:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:24.514 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:02:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:02:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 msgr2=0x7f3b180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 0x7f3b180a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3b341019d0 tx=0x7f3b24010040 comp rx=0 tx=0).stop 2024-04-26T22:02:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 msgr2=0x7f3b3419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00e9c0 tx=0x7f3b1c00ee80 comp rx=0 tx=0).stop 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 shutdown_connections 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f3b180a3cc0 0x7f3b180a6180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b34106db0 0x7f3b341a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b341020b0 0x7f3b3419b0d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 --2- 172.21.15.12:0/1345217696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34068de0 0x7f3b3419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.515+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 >> 172.21.15.12:0/1345217696 conn(0x7f3b340fbef0 msgr2=0x7f3b34102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.516+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 shutdown_connections 2024-04-26T22:02:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.516+0000 7f3b33e00640 1 -- 172.21.15.12:0/1345217696 wait complete. 2024-04-26T22:02:24.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:02:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:24 smithi012 ceph-mon[28184]: pgmap v404: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3815837359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2517982232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:24 smithi154 ceph-mon[32578]: pgmap v404: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:24.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3815837359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2517982232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:24 smithi077 ceph-mon[32280]: pgmap v404: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:02:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3815837359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2517982232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 -- 172.21.15.12:0/1945237737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 msgr2=0x7fdf6c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 --2- 172.21.15.12:0/1945237737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c075810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009930 tx=0x7fdf5402a440 comp rx=0 tx=0).stop 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 -- 172.21.15.12:0/1945237737 shutdown_connections 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 --2- 172.21.15.12:0/1945237737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 0x7fdf6c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 --2- 172.21.15.12:0/1945237737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c075810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 --2- 172.21.15.12:0/1945237737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 0x7fdf6c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 -- 172.21.15.12:0/1945237737 >> 172.21.15.12:0/1945237737 conn(0x7fdf6c0fdcc0 msgr2=0x7fdf6c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 -- 172.21.15.12:0/1945237737 shutdown_connections 2024-04-26T22:02:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.743+0000 7fdf71400640 1 -- 172.21.15.12:0/1945237737 wait complete. 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.744+0000 7fdf71400640 1 Processor -- start 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.744+0000 7fdf71400640 1 -- start start 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 0x7fdf6c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 0x7fdf6c1a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1148a0 con 0x7fdf6c075390 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c114720 con 0x7fdf6c076970 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf71400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c114a20 con 0x7fdf6c10ed20 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34514/0 (socket says 172.21.15.12:34514) 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf6be00640 1 -- 172.21.15.12:0/3747372591 learned_addr learned my addr 172.21.15.12:0/3747372591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf63400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 0x7fdf6c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.745+0000 7fdf70a00640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 0x7fdf6c1a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 -- 172.21.15.12:0/3747372591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 msgr2=0x7fdf6c1a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 0x7fdf6c1a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 -- 172.21.15.12:0/3747372591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 msgr2=0x7fdf6c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 0x7fdf6c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 -- 172.21.15.12:0/3747372591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c1a6c10 con 0x7fdf6c075390 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf6be00640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00ccc0 con 0x7fdf6c075390 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c0108e0 con 0x7fdf6c075390 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c018880 con 0x7fdf6c075390 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c1a6f00 con 0x7fdf6c075390 2024-04-26T22:02:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.746+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a72b0 con 0x7fdf6c075390 2024-04-26T22:02:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.748+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c104ef0 con 0x7fdf6c075390 2024-04-26T22:02:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.750+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf5c010430 con 0x7fdf6c075390 2024-04-26T22:02:24.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.752+0000 7fdf69600640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 0x7fdf500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.752+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdf5c0d0d30 con 0x7fdf6c075390 2024-04-26T22:02:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.752+0000 7fdf63400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 0x7fdf500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.753+0000 7fdf63400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 0x7fdf500a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf54005ab0 tx=0x7fdf54005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.756+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf5c09d1b0 con 0x7fdf6c075390 2024-04-26T22:02:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.963+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf6c02dd50 con 0x7fdf6c075390 2024-04-26T22:02:24.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.968+0000 7fdf69600640 1 -- 172.21.15.12:0/3747372591 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7fdf5c09cab0 con 0x7fdf6c075390 2024-04-26T22:02:24.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:02:24.969 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":94,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:01:44.865520+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:02:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.970+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 msgr2=0x7fdf500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.970+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 0x7fdf500a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf54005ab0 tx=0x7fdf54005a20 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.970+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 msgr2=0x7fdf6c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.970+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00edf0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.970+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 shutdown_connections 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fdf500a3c40 0x7fdf500a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf6c10ed20 0x7fdf6c1a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf6c076970 0x7fdf6c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 --2- 172.21.15.12:0/3747372591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf6c075390 0x7fdf6c19eff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 >> 172.21.15.12:0/3747372591 conn(0x7fdf6c0fdcc0 msgr2=0x7fdf6c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 shutdown_connections 2024-04-26T22:02:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:24.971+0000 7fdf71400640 1 -- 172.21.15.12:0/3747372591 wait complete. 2024-04-26T22:02:24.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:59:43.592535+0000', 'modified': '2024-04-26T22:00:09.736381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24565': {'gid': 24565, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/3734088968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 3734088968}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3734088968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6839/4217479283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 4217479283}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 4217479283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-26T22:02:24.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:02:24.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:02:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.209+0000 7ff5ff400640 1 -- 172.21.15.12:0/1488103621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8106500 msgr2=0x7ff5f8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.209+0000 7ff5ff400640 1 --2- 172.21.15.12:0/1488103621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8106500 0x7ff5f8106980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009900 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 -- 172.21.15.12:0/1488103621 shutdown_connections 2024-04-26T22:02:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 --2- 172.21.15.12:0/1488103621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8106500 0x7ff5f8106980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 --2- 172.21.15.12:0/1488103621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8103b50 0x7ff5f8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 --2- 172.21.15.12:0/1488103621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 0x7ff5f8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 -- 172.21.15.12:0/1488103621 >> 172.21.15.12:0/1488103621 conn(0x7ff5f80fcde0 msgr2=0x7ff5f80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 -- 172.21.15.12:0/1488103621 shutdown_connections 2024-04-26T22:02:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.210+0000 7ff5ff400640 1 -- 172.21.15.12:0/1488103621 wait complete. 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 Processor -- start 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 -- start start 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 0x7ff5f819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8106500 0x7ff5f81a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8114ac0 con 0x7ff5f8103b50 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8114940 con 0x7ff5f81011e0 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.211+0000 7ff5ff400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8114c40 con 0x7ff5f8106500 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34526/0 (socket says 172.21.15.12:34526) 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 -- 172.21.15.12:0/2032834356 learned_addr learned my addr 172.21.15.12:0/2032834356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5fd400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8106500 0x7ff5f81a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 0x7ff5f819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 -- 172.21.15.12:0/2032834356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8106500 msgr2=0x7ff5f81a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8106500 0x7ff5f81a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 -- 172.21.15.12:0/2032834356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 msgr2=0x7ff5f819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 0x7ff5f819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 -- 172.21.15.12:0/2032834356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f81a6d80 con 0x7ff5f8103b50 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.212+0000 7ff5efe00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9f0 tx=0x7ff5e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.213+0000 7ff5f6000640 1 -- 172.21.15.12:0/2032834356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000cd80 con 0x7ff5f8103b50 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.213+0000 7ff5f6000640 1 -- 172.21.15.12:0/2032834356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e00108e0 con 0x7ff5f8103b50 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.213+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f81a7070 con 0x7ff5f8103b50 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.213+0000 7ff5f6000640 1 -- 172.21.15.12:0/2032834356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0018980 con 0x7ff5f8103b50 2024-04-26T22:02:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.213+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f81a75c0 con 0x7ff5f8103b50 2024-04-26T22:02:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.216+0000 7ff602256740 1 Processor -- start 2024-04-26T22:02:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 -- start start 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x5588493525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x5588493550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5588492b1600 con 0x558849352c50 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55884924e100 con 0x55884934dae0 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff602256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5588492ad5c0 con 0x55884934d0f0 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56780/0 (socket says 172.21.15.12:56780) 2024-04-26T22:02:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff5fd400640 1 -- 172.21.15.12:0/1784349338 learned_addr learned my addr 172.21.15.12:0/1784349338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.217+0000 7ff5fca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x5588493525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 -- 172.21.15.12:0/1784349338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 msgr2=0x5588493525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x5588493525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 -- 172.21.15.12:0/1784349338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 msgr2=0x5588493550e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:02:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x5588493550e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 -- 172.21.15.12:0/1784349338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558848fc32d0 con 0x55884934d0f0 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fca00640 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x5588493525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5fd400640 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0033000 tx=0x7ff5f0005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc5099142de110d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5f6000640 1 -- 172.21.15.12:0/2032834356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff5e0010430 con 0x7ff5f8103b50 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.218+0000 7ff5edc00640 1 -- 172.21.15.12:0/1784349338 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0035030 con 0x55884934d0f0 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff5edc00640 1 -- 172.21.15.12:0/1784349338 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f0040070 con 0x55884934d0f0 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff5edc00640 1 -- 172.21.15.12:0/1784349338 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00038e0 con 0x55884934d0f0 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 -- 172.21.15.12:0/1784349338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 msgr2=0x55884934d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0033000 tx=0x7ff5f0005dd0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 -- 172.21.15.12:0/1784349338 shutdown_connections 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x5588493550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x5588493525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 --2- 172.21.15.12:0/1784349338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934d510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.219+0000 7ff602256740 1 -- 172.21.15.12:0/1784349338 >> 172.21.15.12:0/1784349338 conn(0x5588492e1d10 msgr2=0x558849348b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff602256740 1 -- 172.21.15.12:0/1784349338 shutdown_connections 2024-04-26T22:02:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff602256740 1 -- 172.21.15.12:0/1784349338 wait complete. 2024-04-26T22:02:25.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff5f6000640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 0x7ff5dc0a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff5f6000640 1 -- 172.21.15.12:0/2032834356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff5e00d0c00 con 0x7ff5f8103b50 2024-04-26T22:02:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff602256740 1 -- start start 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 0x7ff5dc0a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.220+0000 7ff602256740 1 Processor -- start 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x558849357ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 0x55884935eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558849355ba0 con 0x55884934d0f0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558849238f70 con 0x558849352c50 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558849355d20 con 0x55884934dae0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff602256740 10 client.0 _ll_register_callbacks cb 0x5588492a7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 0x7ff5dc0a6010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x558849349a20 tx=0x7ff5e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5fca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5fca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34538/0 (socket says 172.21.15.12:34538) 2024-04-26T22:02:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5fca00640 1 -- 172.21.15.12:0/2269504977 learned_addr learned my addr 172.21.15.12:0/2269504977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5fd400640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 0x55884935eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.221+0000 7ff5efe00640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x558849357ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 -- 172.21.15.12:0/2269504977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 msgr2=0x558849357ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x558849357ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 -- 172.21.15.12:0/2269504977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 msgr2=0x558849357580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 -- 172.21.15.12:0/2269504977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884908d6b0 con 0x558849352c50 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5fd400640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 0x55884935eb00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5f002a910 tx=0x7ff5f00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8adfba04a399fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5e6000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0052060 con 0x558849352c50 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff5f0003660 con 0x558849352c50 2024-04-26T22:02:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.222+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f0040070 con 0x558849352c50 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff5f0059050 con 0x558849352c50 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff602256740 10 client.25289 fetch_fsmap learned FSMap version 21 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558849361ae0 con 0x558849352c50 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f0040070 con 0x558849352c50 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff5f00459a0 con 0x558849352c50 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.223+0000 7ff602256740 10 client.25289 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff602256740 10 client.25289 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff602256740 20 client.25289 populate_metadata read hostname 'smithi012' 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x558849363c00 0x558849365070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff602256740 4 client.25289 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588492bf210 con 0x558849363c00 2024-04-26T22:02:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.224+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x558849363c00 0x558849365070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.225+0000 7ff5fca00640 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x558849363c00 0x558849365070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74af437a6a32f0d8 server_cookie=39145f25ec94b0d5 in_seq=0 out_seq=0 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.225+0000 7ff5e6000640 10 client.25289 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.225+0000 7ff5e6000640 1 -- 172.21.15.12:0/2269504977 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588492bf210 con 0x558849363c00 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.225+0000 7ff5e6000640 10 client.25289 handle_command_reply: tid=0 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.225+0000 7ff602256740 1 client.25289 shutdown 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 msgr2=0x55884935eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 0x55884935eb00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5f002a910 tx=0x7ff5f00042f0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 shutdown_connections 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x558849363c00 0x558849365070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558849352c50 0x55884935eb00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x558849357ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 --2- 172.21.15.12:0/2269504977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55884934d0f0 0x558849357580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff5e6000640 0 client.25289 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 >> 172.21.15.12:0/2269504977 conn(0x5588492e1d10 msgr2=0x558849347eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 shutdown_connections 2024-04-26T22:02:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 1 -- 172.21.15.12:0/2269504977 wait complete. 2024-04-26T22:02:25.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.226+0000 7ff602256740 20 client.25289 trim_cache size 0 max 16384 2024-04-26T22:02:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.245+0000 7ff602256740 1 Processor -- start 2024-04-26T22:02:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.245+0000 7ff602256740 1 -- start start 2024-04-26T22:02:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5588493633b0 con 0x558849352c50 2024-04-26T22:02:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558849363a80 con 0x55884934dae0 2024-04-26T22:02:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff602256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558849364150 con 0x55884934d0f0 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5fca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52578/0 (socket says 172.21.15.12:52578) 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 -- 172.21.15.12:0/969916784 learned_addr learned my addr 172.21.15.12:0/969916784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5fca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56792/0 (socket says 172.21.15.12:56792) 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5fd400640 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 -- 172.21.15.12:0/969916784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 msgr2=0x55884934efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 -- 172.21.15.12:0/969916784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 msgr2=0x55884935ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5efe00640 1 -- 172.21.15.12:0/969916784 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558849355c60 con 0x55884934dae0 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.246+0000 7ff5fca00640 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.247+0000 7ff5efe00640 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c6a0 tx=0x7ff5e000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3809061c50bd1684 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.247+0000 7ff5ed200640 1 -- 172.21.15.12:0/969916784 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a0d90 con 0x55884934dae0 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.247+0000 7ff5ed200640 1 -- 172.21.15.12:0/969916784 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e00a5650 con 0x55884934dae0 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.247+0000 7ff5ed200640 1 -- 172.21.15.12:0/969916784 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00bbe00 con 0x55884934dae0 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- 172.21.15.12:0/969916784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 msgr2=0x55884934f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c6a0 tx=0x7ff5e000cf20 comp rx=0 tx=0).stop 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- 172.21.15.12:0/969916784 shutdown_connections 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934dae0 0x55884934f520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 --2- 172.21.15.12:0/969916784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934d0f0 0x55884934efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- 172.21.15.12:0/969916784 >> 172.21.15.12:0/969916784 conn(0x5588492e1d10 msgr2=0x5588493485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- 172.21.15.12:0/969916784 shutdown_connections 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- 172.21.15.12:0/969916784 wait complete. 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.248+0000 7ff602256740 1 -- start start 2024-04-26T22:02:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 Processor -- start 2024-04-26T22:02:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934d0f0 0x55884935cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x55884935d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558849364ef0 con 0x558849352c50 2024-04-26T22:02:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558849364820 con 0x55884934d0f0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5588493655c0 con 0x55884934dae0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff602256740 10 client.0 _ll_register_callbacks cb 0x7ff5e8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x55884935d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34554/0 (socket says 172.21.15.12:34554) 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff5fd400640 1 -- 172.21.15.12:0/3361842281 learned_addr learned my addr 172.21.15.12:0/3361842281 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.249+0000 7ff5fca00640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934d0f0 0x55884935cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 -- 172.21.15.12:0/3361842281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 msgr2=0x55884935d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x55884935d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 -- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934d0f0 msgr2=0x55884935cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934d0f0 0x55884935cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 -- 172.21.15.12:0/3361842281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884935df50 con 0x558849352c50 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5fd400640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0045d70 tx=0x7ff5f002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb25237219f3d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5e6000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0062060 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e00a5650 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.250+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff5f0045670 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff5f0052070 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 10 client.15183 fetch_fsmap learned FSMap version 21 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5588493c9ea0 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e00a5650 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff5f00039e0 con 0x558849352c50 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 10 client.15183 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 10 client.15183 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 20 client.15183 populate_metadata read hostname 'smithi012' 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5588493ceab0 0x5588493cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 4 client.15183 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:02:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.251+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "status"}) -- 0x5588492b9080 con 0x5588493ceab0 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.252+0000 7ff5fca00640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5588493ceab0 0x5588493cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.252+0000 7ff5fca00640 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5588493ceab0 0x5588493cd7c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942d0dda631f3432 server_cookie=a2350b9b8848a106 in_seq=0 out_seq=0 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.252+0000 7ff5e6000640 10 client.15183 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff5e6000640 1 -- 172.21.15.12:0/3361842281 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5588492b9080 con 0x5588493ceab0 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff5e6000640 10 client.15183 handle_command_reply: tid=0 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 client.15183 shutdown 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 msgr2=0x55884935d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0045d70 tx=0x7ff5f002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 shutdown_connections 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5588493ceab0 0x5588493cd7c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff5e6000640 0 client.15183 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558849352c50 0x55884935d720 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55884934dae0 0x55884935d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 --2- 172.21.15.12:0/3361842281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884934d0f0 0x55884935cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 >> 172.21.15.12:0/3361842281 conn(0x5588492e1d10 msgr2=0x558849357540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 shutdown_connections 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 1 -- 172.21.15.12:0/3361842281 wait complete. 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.253+0000 7ff602256740 20 client.15183 trim_cache size 0 max 16384 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "d6d929b6-0416-11ef-bc93-c7b262605968", 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24581, 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-26T22:02:25.258 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 161.18398263500001, 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 21, 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 94, 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 167.65703714099999 2024-04-26T22:02:25.259 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 msgr2=0x7ff5dc0a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 0x7ff5dc0a6010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x558849349a20 tx=0x7ff5e8009500 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 msgr2=0x7ff5f819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9f0 tx=0x7ff5e000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 shutdown_connections 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7ff5dc0a3b50 0x7ff5dc0a6010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5f8106500 0x7ff5f81a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5f8103b50 0x7ff5f819f6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 --2- 172.21.15.12:0/2032834356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5f81011e0 0x7ff5f819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 >> 172.21.15.12:0/2032834356 conn(0x7ff5f80fcde0 msgr2=0x7ff5f8104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 shutdown_connections 2024-04-26T22:02:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.254+0000 7ff5ff400640 1 -- 172.21.15.12:0/2032834356 wait complete. 2024-04-26T22:02:25.274 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T22:02:25.283 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T22:02:25.284 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T22:02:25.284 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T22:02:25.284 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T22:02:25.285 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:02:25.285 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:02:25.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T22:02:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 -- 172.21.15.12:0/2185359037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f0108fc0 msgr2=0x7f58f01093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 --2- 172.21.15.12:0/2185359037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f0108fc0 0x7f58f01093e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009900 tx=0x7f58e002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 -- 172.21.15.12:0/2185359037 shutdown_connections 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 --2- 172.21.15.12:0/2185359037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f0111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 --2- 172.21.15.12:0/2185359037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f010a1b0 0x7f58f010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 --2- 172.21.15.12:0/2185359037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f0108fc0 0x7f58f01093e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 -- 172.21.15.12:0/2185359037 >> 172.21.15.12:0/2185359037 conn(0x7f58f0077f80 msgr2=0x7f58f007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 -- 172.21.15.12:0/2185359037 shutdown_connections 2024-04-26T22:02:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.492+0000 7f58f5200640 1 -- 172.21.15.12:0/2185359037 wait complete. 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.493+0000 7f58f5200640 1 Processor -- start 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 -- start start 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f0108fc0 0x7f58f019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 0x7f58f019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0114b10 con 0x7f58f010a1b0 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0114990 con 0x7f58f010ab70 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58f5200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0114c90 con 0x7f58f0108fc0 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58ee800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58e5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 0x7f58f019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58ee800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52598/0 (socket says 172.21.15.12:52598) 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58ee800640 1 -- 172.21.15.12:0/1133655250 learned_addr learned my addr 172.21.15.12:0/1133655250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.494+0000 7f58ede00640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f0108fc0 0x7f58f019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 -- 172.21.15.12:0/1133655250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f0108fc0 msgr2=0x7f58f019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f0108fc0 0x7f58f019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 -- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 msgr2=0x7f58f019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 0x7f58f019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 -- 172.21.15.12:0/1133655250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a6da0 con 0x7f58f010ab70 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58e5400640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 0x7f58f019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58ee800640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58e800eb10 tx=0x7f58e800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.495+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8004b20 con 0x7f58f010ab70 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.496+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a7090 con 0x7f58f010ab70 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.496+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8004cc0 con 0x7f58f010ab70 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.496+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8010670 con 0x7f58f010ab70 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.496+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a7590 con 0x7f58f010ab70 2024-04-26T22:02:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.497+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f01102e0 con 0x7f58f010ab70 2024-04-26T22:02:25.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.499+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58e8005020 con 0x7f58f010ab70 2024-04-26T22:02:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.500+0000 7f58e7400640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 0x7f58d40a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.500+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f58e80d0f40 con 0x7f58f010ab70 2024-04-26T22:02:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.500+0000 7f58ede00640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 0x7f58d40a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.501+0000 7f58ede00640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 0x7f58d40a61b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58e0005ab0 tx=0x7f58e0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.504+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f58e809d3c0 con 0x7f58f010ab70 2024-04-26T22:02:25.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:25 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1345217696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:25 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3747372591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:25 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1345217696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:25 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3747372591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:25 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1345217696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:25 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3747372591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:02:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:25.768+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f58f0063530 con 0x7f58f010ab70 2024-04-26T22:02:26.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.370+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f58e8099330 con 0x7f58f010ab70 2024-04-26T22:02:26.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.372+0000 7f58e7400640 1 -- 172.21.15.12:0/1133655250 <== mon.2 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f58e8078680 con 0x7f58f010ab70 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 msgr2=0x7f58d40a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 0x7f58d40a61b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58e0005ab0 tx=0x7f58e0005a20 comp rx=0 tx=0).stop 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 msgr2=0x7f58f01a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58e800eb10 tx=0x7f58e800c2d0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 shutdown_connections 2024-04-26T22:02:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f58d40a3cf0 0x7f58d40a61b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58f010ab70 0x7f58f01a6730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f010a1b0 0x7f58f019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 --2- 172.21.15.12:0/1133655250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f0108fc0 0x7f58f019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 >> 172.21.15.12:0/1133655250 conn(0x7f58f0077f80 msgr2=0x7f58f010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 shutdown_connections 2024-04-26T22:02:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.379+0000 7f58f5200640 1 -- 172.21.15.12:0/1133655250 wait complete. 2024-04-26T22:02:26.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T22:02:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 -- 172.21.15.12:0/1610848460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc10a840 msgr2=0x7f77dc10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 --2- 172.21.15.12:0/1610848460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc10a840 0x7f77dc10cce0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009860 tx=0x7f77d402a3d0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 -- 172.21.15.12:0/1610848460 shutdown_connections 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 --2- 172.21.15.12:0/1610848460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc10a840 0x7f77dc10cce0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 --2- 172.21.15.12:0/1610848460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc075330 0x7f77dc075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 --2- 172.21.15.12:0/1610848460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc076910 0x7f77dc074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 -- 172.21.15.12:0/1610848460 >> 172.21.15.12:0/1610848460 conn(0x7f77dc0fdf20 msgr2=0x7f77dc100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 -- 172.21.15.12:0/1610848460 shutdown_connections 2024-04-26T22:02:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.592+0000 7f77e2a00640 1 -- 172.21.15.12:0/1610848460 wait complete. 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.593+0000 7f77e2a00640 1 Processor -- start 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.593+0000 7f77e2a00640 1 -- start start 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 0x7f77dc19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc10a840 0x7f77dc1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110730 con 0x7f77dc076910 2024-04-26T22:02:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1105b0 con 0x7f77dc10a840 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e2a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1108b0 con 0x7f77dc075330 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:56850/0 (socket says 172.21.15.12:56850) 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 -- 172.21.15.12:0/770639818 learned_addr learned my addr 172.21.15.12:0/770639818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77e0a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc10a840 0x7f77dc1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77d3400640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 0x7f77dc19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc10a840 msgr2=0x7f77dc1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc10a840 0x7f77dc1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 msgr2=0x7f77dc19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 0x7f77dc19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77dbe00640 1 -- 172.21.15.12:0/770639818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc1a2940 con 0x7f77dc075330 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.594+0000 7f77d3400640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 0x7f77dc19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77dbe00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00eb90 tx=0x7f77cc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc004d20 con 0x7f77dc075330 2024-04-26T22:02:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc00ee90 con 0x7f77dc075330 2024-04-26T22:02:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc010670 con 0x7f77dc075330 2024-04-26T22:02:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc1a2c30 con 0x7f77dc075330 2024-04-26T22:02:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.595+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc1a3130 con 0x7f77dc075330 2024-04-26T22:02:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.597+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc109600 con 0x7f77dc075330 2024-04-26T22:02:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.599+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f77cc005020 con 0x7f77dc075330 2024-04-26T22:02:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.600+0000 7f77d9600640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 0x7f77c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.600+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f77cc0d10d0 con 0x7f77dc075330 2024-04-26T22:02:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.600+0000 7f77d3400640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 msgr2=0x7f77c00a6280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.600+0000 7f77d3400640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 0x7f77c00a6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:02:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.604+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f77cc09d550 con 0x7f77dc075330 2024-04-26T22:02:26.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-mon[32578]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:26.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1133655250' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: ignoring --setuser ceph since I am not root 2024-04-26T22:02:26.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: ignoring --setgroup ceph since I am not root 2024-04-26T22:02:26.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-mgr[34447]: -- 172.21.15.154:0/868736933 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624e85205a0 con 0x5624e84f5000 2024-04-26T22:02:26.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:26.538+0000 7fab35946200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:02:26.705 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setuser ceph since I am not root 2024-04-26T22:02:26.705 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: ignoring --setgroup ceph since I am not root 2024-04-26T22:02:26.705 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:26.703+0000 7fde89163200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:02:26.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-mon[28184]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:26.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1133655250' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.706 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.719 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-mon[32280]: pgmap v405: 129 pgs: 129 active+clean; 632 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:26.719 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1133655250' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.719 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T22:02:26.720 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: ignoring --setuser ceph since I am not root 2024-04-26T22:02:26.720 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: ignoring --setgroup ceph since I am not root 2024-04-26T22:02:26.720 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-mgr[33682]: -- 172.21.15.77:0/2542564730 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562cb233c5a0 con 0x562cb2311000 2024-04-26T22:02:26.720 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:26.533+0000 7fd11a16f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:02:26.720 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:26.718+0000 7fd11a16f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.801+0000 7f77d3400640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 msgr2=0x7f77c00a6280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.801+0000 7f77d3400640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 0x7f77c00a6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:02:26.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.816+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f77dc02dad0 con 0x7f77dc075330 2024-04-26T22:02:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.826+0000 7f77d9600640 1 -- 172.21.15.12:0/770639818 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f77cc09ce50 con 0x7f77dc075330 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.828+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 msgr2=0x7f77c00a6280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.828+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 0x7f77c00a6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.828+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 msgr2=0x7f77dc19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.828+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00eb90 tx=0x7f77cc00c6a0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 shutdown_connections 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f77c00a3dc0 0x7f77c00a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77dc10a840 0x7f77dc1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77dc076910 0x7f77dc19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 --2- 172.21.15.12:0/770639818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77dc075330 0x7f77dc19ad30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 >> 172.21.15.12:0/770639818 conn(0x7f77dc0fdf20 msgr2=0x7f77dc1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 shutdown_connections 2024-04-26T22:02:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:26.829+0000 7f77e2a00640 1 -- 172.21.15.12:0/770639818 wait complete. 2024-04-26T22:02:26.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T22:02:26.982 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:26.821+0000 7fd11a16f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:02:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 -- 172.21.15.12:0/3676355445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588102850 msgr2=0x7fd588102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 --2- 172.21.15.12:0/3676355445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588102850 0x7fd588102c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd578009ca0 tx=0x7fd57802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 -- 172.21.15.12:0/3676355445 shutdown_connections 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 --2- 172.21.15.12:0/3676355445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd58810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 --2- 172.21.15.12:0/3676355445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588103a40 0x7fd588103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 --2- 172.21.15.12:0/3676355445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588102850 0x7fd588102c70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 -- 172.21.15.12:0/3676355445 >> 172.21.15.12:0/3676355445 conn(0x7fd5880fddc0 msgr2=0x7fd588100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.031+0000 7fd58ca00640 1 -- 172.21.15.12:0/3676355445 shutdown_connections 2024-04-26T22:02:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.032+0000 7fd58ca00640 1 -- 172.21.15.12:0/3676355445 wait complete. 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.032+0000 7fd58ca00640 1 Processor -- start 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 -- start start 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 0x7fd5880766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588103a40 0x7fd588074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810e330 con 0x7fd588104400 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810e1b0 con 0x7fd588102850 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd58ca00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810e4b0 con 0x7fd588103a40 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd587200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd586800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 0x7fd5880766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd586800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 0x7fd5880766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52626/0 (socket says 172.21.15.12:52626) 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd587200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34626/0 (socket says 172.21.15.12:34626) 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd57de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588103a40 0x7fd588074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.033+0000 7fd586800640 1 -- 172.21.15.12:0/2648819080 learned_addr learned my addr 172.21.15.12:0/2648819080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588103a40 msgr2=0x7fd588074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588103a40 0x7fd588074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 msgr2=0x7fd5880766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 0x7fd5880766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 -- 172.21.15.12:0/2648819080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd588075af0 con 0x7fd588104400 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd587200640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e970 tx=0x7fd58000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58000ccc0 con 0x7fd588104400 2024-04-26T22:02:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881a2b10 con 0x7fd588104400 2024-04-26T22:02:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd580005020 con 0x7fd588104400 2024-04-26T22:02:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580004a00 con 0x7fd588104400 2024-04-26T22:02:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.034+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a3060 con 0x7fd588104400 2024-04-26T22:02:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.036+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588109c40 con 0x7fd588104400 2024-04-26T22:02:27.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.038+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7fd580004ba0 con 0x7fd588104400 2024-04-26T22:02:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.039+0000 7fd57fe00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 0x7fd56c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.039+0000 7fd586800640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 msgr2=0x7fd56c0a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.039+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd5800d0940 con 0x7fd588104400 2024-04-26T22:02:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.039+0000 7fd586800640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 0x7fd56c0a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:02:27.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.044+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd58009ce20 con 0x7fd588104400 2024-04-26T22:02:27.136 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:26.725+0000 7fab35946200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:27.136 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:26.828+0000 7fab35946200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:02:27.136 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:27.136+0000 7fab35946200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:02:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.240+0000 7fd586800640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 msgr2=0x7fd56c0a6140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.240+0000 7fd586800640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 0x7fd56c0a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:02:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.269+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd588063640 con 0x7fd588104400 2024-04-26T22:02:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.277+0000 7fd57fe00640 1 -- 172.21.15.12:0/2648819080 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fd58009c720 con 0x7fd588104400 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.279+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 msgr2=0x7fd56c0a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.279+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 0x7fd56c0a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 msgr2=0x7fd588075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e970 tx=0x7fd58000ee30 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 shutdown_connections 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fd56c0a3c80 0x7fd56c0a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd588104400 0x7fd588075260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd588103a40 0x7fd588074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 --2- 172.21.15.12:0/2648819080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd588102850 0x7fd5880766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 >> 172.21.15.12:0/2648819080 conn(0x7fd5880fddc0 msgr2=0x7fd588106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 shutdown_connections 2024-04-26T22:02:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:27.280+0000 7fd58ca00640 1 -- 172.21.15.12:0/2648819080 wait complete. 2024-04-26T22:02:27.295 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:26.959+0000 7fde89163200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:27.295 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:27.097+0000 7fde89163200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:02:27.298 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T22:02:27.315 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T22:02:27.315 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T22:02:27.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:27.315 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-26T22:02:27.366 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T22:02:27.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:27.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T22:02:27.390 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:27.220+0000 7fab35946200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:02:27.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:02:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-mon[32578]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T22:02:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' 2024-04-26T22:02:27.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2648819080' entity='client.admin' 2024-04-26T22:02:27.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:27.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-26T22:02:27.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:27.128+0000 7fd11a16f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:02:27.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:27.213+0000 7fd11a16f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:02:27.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:02:27.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-mon[32280]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T22:02:27.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' 2024-04-26T22:02:27.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2648819080' entity='client.admin' 2024-04-26T22:02:27.548 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:27.522+0000 7fde89163200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:02:27.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T22:02:27.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-mon[28184]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T22:02:27.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' 2024-04-26T22:02:27.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2648819080' entity='client.admin' 2024-04-26T22:02:27.553 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T22:02:27.553 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:02:27.553 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:27.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T22:02:27.822 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:27.641+0000 7fde89163200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:02:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:02:28.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:28.387+0000 7fd11a16f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:02:28.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:28.557+0000 7fd11a16f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:28.745 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:28.402+0000 7fab35946200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:02:28.745 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:28.571+0000 7fab35946200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:28.745 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:28.745+0000 7fab35946200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:02:29.019 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:28.732+0000 7fd11a16f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:02:29.314 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:29.018+0000 7fd11a16f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:02:29.315 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:29.111+0000 7fd11a16f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:02:29.329 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:29.033+0000 7fab35946200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:02:29.329 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:29.125+0000 7fab35946200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:02:29.329 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:29.329+0000 7fab35946200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:02:29.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:29.280+0000 7fde89163200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:02:29.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:29.520+0000 7fde89163200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:29.640 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:29.523+0000 7fab35946200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:29.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:29.314+0000 7fd11a16f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:02:29.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:29.508+0000 7fd11a16f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:29.902 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:29.744+0000 7fde89163200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:02:30.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 -- 172.21.15.12:0/3156669315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c1094a0 msgr2=0x7f848c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 --2- 172.21.15.12:0/3156669315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c1094a0 0x7f848c109880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f847c00b0a0 tx=0x7f847c00b3b0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 -- 172.21.15.12:0/3156669315 shutdown_connections 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 --2- 172.21.15.12:0/3156669315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c06bad0 0x7f848c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 --2- 172.21.15.12:0/3156669315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c1094a0 0x7f848c109880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 --2- 172.21.15.12:0/3156669315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06b130 0x7f848c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 -- 172.21.15.12:0/3156669315 >> 172.21.15.12:0/3156669315 conn(0x7f848c0792d0 msgr2=0x7f848c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 -- 172.21.15.12:0/3156669315 shutdown_connections 2024-04-26T22:02:30.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.381+0000 7f8492c00700 1 -- 172.21.15.12:0/3156669315 wait complete. 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.382+0000 7f8492c00700 1 Processor -- start 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.382+0000 7f8492c00700 1 -- start start 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06bad0 0x7f848c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c1094a0 0x7f848c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f848c1135e0 con 0x7f848c06b130 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f848c113460 con 0x7f848c1094a0 2024-04-26T22:02:30.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8492c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f848c113760 con 0x7f848c06bad0 2024-04-26T22:02:30.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f848be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f848b400700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06bad0 0x7f848c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f848be00700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38110/0 (socket says 172.21.15.12:38110) 2024-04-26T22:02:30.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f848be00700 1 -- 172.21.15.12:0/3255316733 learned_addr learned my addr 172.21.15.12:0/3255316733 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.383+0000 7f8490c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c1094a0 0x7f848c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06bad0 msgr2=0x7f848c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06bad0 0x7f848c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c1094a0 msgr2=0x7f848c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c1094a0 0x7f848c102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 -- 172.21.15.12:0/3255316733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c1a8080 con 0x7f848c06b130 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f848be00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f847c00d010 tx=0x7f847c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c021070 con 0x7f848c06b130 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f847c007e30 con 0x7f848c06b130 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c1a8310 con 0x7f848c06b130 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.384+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0045f0 con 0x7f848c06b130 2024-04-26T22:02:30.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.385+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c1a8810 con 0x7f848c06b130 2024-04-26T22:02:30.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.386+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c06a110 con 0x7f848c06b130 2024-04-26T22:02:30.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.389+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138714+0+0 (secure 0 0 0) 0x7f847c003d20 con 0x7f848c06b130 2024-04-26T22:02:30.390 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:30.128+0000 7fab35946200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:02:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.390+0000 7f8488c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 0x7f84700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.391+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f847c00f020 con 0x7f848c06b130 2024-04-26T22:02:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.391+0000 7f848b400700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 msgr2=0x7f84700a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.391+0000 7f848b400700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 0x7f84700a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:02:30.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.399+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f847c02a070 con 0x7f848c06b130 2024-04-26T22:02:30.482 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:30.106+0000 7fd11a16f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:02:30.561 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:30.154+0000 7fde89163200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:02:30.561 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:30.288+0000 7fde89163200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:02:30.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.592+0000 7f848b400700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 msgr2=0x7f84700a6260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:30.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.592+0000 7f848b400700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 0x7f84700a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:02:30.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.673+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f848c02e490 con 0x7f848c06b130 2024-04-26T22:02:30.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.674+0000 7f8488c00700 1 -- 172.21.15.12:0/3255316733 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7f847c09dc00 con 0x7f848c06b130 2024-04-26T22:02:30.675 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T22:02:30.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.676+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 msgr2=0x7f84700a6260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 0x7f84700a6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 msgr2=0x7f848c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f847c00d010 tx=0x7f847c004d00 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 shutdown_connections 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c1094a0 0x7f848c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c06b130 0x7f848c1023f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7f84700a3da0 0x7f84700a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 --2- 172.21.15.12:0/3255316733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f848c06bad0 0x7f848c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 >> 172.21.15.12:0/3255316733 conn(0x7f848c0792d0 msgr2=0x7f848c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 shutdown_connections 2024-04-26T22:02:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:30.677+0000 7f8492c00700 1 -- 172.21.15.12:0/3255316733 wait complete. 2024-04-26T22:02:30.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:30.559+0000 7fde89163200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:02:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:30 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3255316733' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:02:30.826 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:30 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3255316733' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:02:30.826 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:30.526+0000 7fd11a16f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:02:30.826 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:30.623+0000 7fd11a16f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:02:30.850 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:30 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3255316733' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T22:02:30.850 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:30.550+0000 7fab35946200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:02:30.850 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:30.647+0000 7fab35946200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:02:31.085 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:30.830+0000 7fde89163200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:02:31.232 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:30.826+0000 7fd11a16f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:02:31.458 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:02:31.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:31.459 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-04-26T22:02:31.508 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:02:31.508 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:02:31.508 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:02:31.508 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 41972sec preferred_lft 41972sec 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-04-26T22:02:31.509 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:02:31.510 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:02:31.510 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T22:02:31.510 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-04-26T22:02:31.511 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:02:31.511 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T22:02:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:31.640 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:30.849+0000 7fab35946200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:02:31.640 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:31.377+0000 7fab35946200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:02:31.641 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:31.468+0000 7fab35946200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:02:31.692 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:31.354+0000 7fd11a16f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:02:31.692 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:31.448+0000 7fd11a16f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:02:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:31.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:31.748 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:02:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-04-26T22:02:31.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:31.811 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-04-26T22:02:31.849 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-04-26T22:02:31.850 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:02:31.850 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T22:02:31.850 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T22:02:31.850 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T22:02:31.850 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T22:02:31.982 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:31.692+0000 7fd11a16f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:02:31.982 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:31.824+0000 7fd11a16f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:02:31.982 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:31.920+0000 7fd11a16f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:02:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:32.010 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:31.676+0000 7fde89163200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:02:32.060 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:31.713+0000 7fab35946200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:02:32.060 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:31.845+0000 7fab35946200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:02:32.060 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:31.942+0000 7fab35946200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:02:32.060 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:32.059+0000 7fab35946200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:32.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:32.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:32.104 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-26T22:02:32.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:32.146 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-04-26T22:02:32.229 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:02:32.229 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T22:02:32.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:02:32.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T22:02:32.229 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T22:02:32.338 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:32.036+0000 7fd11a16f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:32.448 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T22:02:32.448 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T22:02:32.449 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T22:02:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:32.554 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:32.286+0000 7fde89163200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:02:32.554 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:32.432+0000 7fde89163200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:02:32.640 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:32.363+0000 7fab35946200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:02:32.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:32.721 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:02:32.721 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-26T22:02:32.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-04-26T22:02:32.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:02:32.721 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-26T22:02:32.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:32.338+0000 7fd11a16f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:02:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:32.822 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:32.736+0000 7fde89163200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:02:32.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:32.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:02:32.908 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:32.945 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:02:32.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:02:32.946 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:33.033 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T22:02:33.033 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T22:02:33.033 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/6437fa6c-391f-4fc8-92f8-759348360930 2024-04-26T22:02:33.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:33.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/6437fa6c-391f-4fc8-92f8-759348360930 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T22:02:33.181 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T22:02:33.181 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-26T22:02:33.218 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:32.935+0000 7fd11a16f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:02:33.218 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y[33678]: 2024-04-26T22:02:33.148+0000 7fd11a16f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:02:33.390 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:32.968+0000 7fab35946200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z[34443]: 2024-04-26T22:02:33.185+0000 7fab35946200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: Standby manager daemon y restarted 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: Standby manager daemon y started 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: Standby manager daemon z restarted 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: Standby manager daemon z started 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:02:33.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:33 smithi154 ceph-mon[32578]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: Standby manager daemon y restarted 2024-04-26T22:02:33.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: Standby manager daemon y started 2024-04-26T22:02:33.437 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: Standby manager daemon z restarted 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: Standby manager daemon z started 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:02:33.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-mon[28184]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: Standby manager daemon y restarted 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: Standby manager daemon y started 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.77:0/3606264644' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: Standby manager daemon z restarted 2024-04-26T22:02:33.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: Standby manager daemon z started 2024-04-26T22:02:33.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T22:02:33.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T22:02:33.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T22:02:33.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:33 smithi077 ceph-mon[32280]: from='mgr.? 172.21.15.154:0/3512697684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T22:02:33.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:33.446+0000 7fde89163200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:02:33.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:33.577+0000 7fde89163200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:02:33.971 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-26T22:02:33.972 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-26T22:02:33.972 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.77:3300,172.21.15.154:3300" 2024-04-26T22:02:33.972 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with new device syntax: 0@d6d929b6-0416-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/6437fa6c-391f-4fc8-92f8-759348360930 2024-04-26T22:02:33.972 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.12:3300/172.21.15.77:3300/172.21.15.154:3300" will pass to kernel 2024-04-26T22:02:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:02:33.974 INFO:teuthology.orchestra.run.smithi012.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:02:33.975 INFO:teuthology.orchestra.run.smithi012.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:02:33.975 INFO:teuthology.orchestra.run.smithi012.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:02:33.975 INFO:teuthology.orchestra.run.smithi012.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:02:33.979 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:02:33.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:33.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:34.048 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:02:34.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:02:34.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:34.158 INFO:teuthology.orchestra.run.smithi012.stdout:client25337 2024-04-26T22:02:34.159 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:02:34.159 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:02:34.208 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:02:34.219 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:33.918+0000 7fde89163200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:02:34.219 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:34.108+0000 7fde89163200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:02:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:02:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:34.316 INFO:teuthology.orchestra.run.smithi012.stdout:client25337 2024-04-26T22:02:34.316 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:02:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:02:34.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:34.476 INFO:teuthology.orchestra.run.smithi012.stdout:client25337 2024-04-26T22:02:34.477 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6d929b6-0416-11ef-bc93-c7b262605968.client25337/status') 2024-04-26T22:02:34.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:34 smithi077 ceph-mon[32280]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T22:02:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:02:34.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:34 smithi012 ceph-mon[28184]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T22:02:34.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:34.244+0000 7fde89163200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:02:34.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:34.402+0000 7fde89163200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:02:34.600 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.25337 (3)172.21.15.12:0/3375406635 2024-04-26T22:02:34.600 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-04-26T22:02:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-04-26T22:02:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-04-26T22:02:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:71 bytes copied, 0.000105449 s, 673 kB/s 2024-04-26T22:02:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:02:34.612 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T22:02:34.623 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T22:02:34.633 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:02:34.633 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:02:34.634 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T22:02:34.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:34 smithi154 ceph-mon[32578]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T22:02:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 -- 172.21.15.12:0/4128102314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe941027c0 msgr2=0x7fbe94102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 --2- 172.21.15.12:0/4128102314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe941027c0 0x7fbe94102be0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009900 tx=0x7fbe8402a400 comp rx=0 tx=0).stop 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 -- 172.21.15.12:0/4128102314 shutdown_connections 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 --2- 172.21.15.12:0/4128102314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe94104370 0x7fbe9410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 --2- 172.21.15.12:0/4128102314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe941039b0 0x7fbe94103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 --2- 172.21.15.12:0/4128102314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe941027c0 0x7fbe94102be0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.830+0000 7fbe93e00640 1 -- 172.21.15.12:0/4128102314 >> 172.21.15.12:0/4128102314 conn(0x7fbe940fdd30 msgr2=0x7fbe94100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.831+0000 7fbe93e00640 1 -- 172.21.15.12:0/4128102314 shutdown_connections 2024-04-26T22:02:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.831+0000 7fbe93e00640 1 -- 172.21.15.12:0/4128102314 wait complete. 2024-04-26T22:02:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.832+0000 7fbe93e00640 1 Processor -- start 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.832+0000 7fbe93e00640 1 -- start start 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.832+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 0x7fbe9419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.832+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe941039b0 0x7fbe9419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe94104370 0x7fbe941a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe93e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e330 con 0x7fbe94104370 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe93e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e1b0 con 0x7fbe941027c0 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe93e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9410e4b0 con 0x7fbe941039b0 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 0x7fbe9419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 0x7fbe9419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:49140/0 (socket says 172.21.15.12:49140) 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 -- 172.21.15.12:0/3677294486 learned_addr learned my addr 172.21.15.12:0/3677294486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe93400640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe94104370 0x7fbe941a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe941039b0 0x7fbe9419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe941039b0 msgr2=0x7fbe9419b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe941039b0 0x7fbe9419b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe94104370 msgr2=0x7fbe941a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe94104370 0x7fbe941a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.833+0000 7fbe92a00640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe941a28b0 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe92a00640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 0x7fbe9419ac70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe84031500 tx=0x7fbe84003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8403f070 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe941a2b40 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe84031c50 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84035030 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.834+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a3080 con 0x7fbe941027c0 2024-04-26T22:02:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.835+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe941a3250 con 0x7fbe941027c0 2024-04-26T22:02:34.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.837+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 138714+0+0 (secure 0 0 0) 0x7fbe84042070 con 0x7fbe941027c0 2024-04-26T22:02:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.839+0000 7fbe8b400640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 0x7fbe780a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.839+0000 7fbe92000640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 msgr2=0x7fbe780a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.839+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbe84070020 con 0x7fbe941027c0 2024-04-26T22:02:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.839+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 0x7fbe780a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:02:34.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:34.841+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe840bdd20 con 0x7fbe941027c0 2024-04-26T22:02:35.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.040+0000 7fbe92000640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 msgr2=0x7fbe780a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:35.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.040+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 0x7fbe780a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T22:02:35.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.070+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbe941a3420 con 0x7fbe780a3d00 2024-04-26T22:02:35.072 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:34.792+0000 7fde89163200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:02:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.441+0000 7fbe92000640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 msgr2=0x7fbe780a61c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/3660687894 2024-04-26T22:02:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.441+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 0x7fbe780a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T22:02:35.903 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:35.599+0000 7fde89163200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:02:35.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.930+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7fbe840b9c90 con 0x7fbe941027c0 2024-04-26T22:02:35.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.931+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 msgr2=0x7fbe780a61c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:02:35.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:35.931+0000 7fbe8b400640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/3660687894,v1:172.21.15.12:6801/3660687894] conn(0x7fbe780a3d00 0x7fbe780a61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:36.162 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:35.901+0000 7fde89163200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: Active manager daemon x restarted 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: Activating manager daemon x 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: mgrmap e20: x(active, starting, since 0.0260506s), standbys: y, z 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:02:36.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:02:36.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:02:36.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:35 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:02:36.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: Active manager daemon x restarted 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: Activating manager daemon x 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: mgrmap e20: x(active, starting, since 0.0260506s), standbys: y, z 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:02:36.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:02:36.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:02:36.235 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:35 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:02:36.272 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: Active manager daemon x restarted 2024-04-26T22:02:36.272 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: Activating manager daemon x 2024-04-26T22:02:36.272 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T22:02:36.272 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: mgrmap e20: x(active, starting, since 0.0260506s), standbys: y, z 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:02:36.273 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T22:02:36.274 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T22:02:36.275 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:35 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fddd9200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fddd9200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fddd9200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fddd9200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fddd9200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fdddec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fdddec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fdddec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fdddec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:36.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:02:36.251+0000 7fdddec00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:02:37.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.006+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mon.2 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fbe840ef0a0 con 0x7fbe941027c0 2024-04-26T22:02:37.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.008+0000 7fbe8b400640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbe780521b0 0x7fbe78052590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.008+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbe84037ec0 con 0x7fbe780521b0 2024-04-26T22:02:37.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.010+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbe780521b0 0x7fbe78052590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.014+0000 7fbe92000640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbe780521b0 0x7fbe78052590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe9419c170 tx=0x7fbe80005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:37.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.108+0000 7fbe8b400640 1 -- 172.21.15.12:0/3677294486 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbe84037ec0 con 0x7fbe780521b0 2024-04-26T22:02:37.109 INFO:teuthology.orchestra.run.smithi012.stdout:Schedule set for path / 2024-04-26T22:02:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.110+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbe780521b0 msgr2=0x7fbe78052590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.110+0000 7fbe93e00640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbe780521b0 0x7fbe78052590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe9419c170 tx=0x7fbe80005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.110+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 msgr2=0x7fbe9419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.110+0000 7fbe93e00640 1 --2- 172.21.15.12:0/3677294486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe941027c0 0x7fbe9419ac70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe84031500 tx=0x7fbe84003c10 comp rx=0 tx=0).stop 2024-04-26T22:02:37.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.111+0000 7fbe93400640 1 -- 172.21.15.12:0/3677294486 reap_dead start 2024-04-26T22:02:37.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.111+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 shutdown_connections 2024-04-26T22:02:37.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.111+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 >> 172.21.15.12:0/3677294486 conn(0x7fbe940fdd30 msgr2=0x7fbe94074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:37.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.111+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 shutdown_connections 2024-04-26T22:02:37.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.111+0000 7fbe93e00640 1 -- 172.21.15.12:0/3677294486 wait complete. 2024-04-26T22:02:37.132 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: Manager daemon x is now available 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:36 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 -- 172.21.15.12:0/2682764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3901016b0 msgr2=0x7fc390101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 --2- 172.21.15.12:0/2682764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3901016b0 0x7fc390101ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 -- 172.21.15.12:0/2682764653 shutdown_connections 2024-04-26T22:02:37.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 --2- 172.21.15.12:0/2682764653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc390103260 0x7fc390109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 --2- 172.21.15.12:0/2682764653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901028a0 0x7fc390102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 --2- 172.21.15.12:0/2682764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3901016b0 0x7fc390101ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 -- 172.21.15.12:0/2682764653 >> 172.21.15.12:0/2682764653 conn(0x7fc3900fcc20 msgr2=0x7fc3900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 -- 172.21.15.12:0/2682764653 shutdown_connections 2024-04-26T22:02:37.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.313+0000 7fc396200640 1 -- 172.21.15.12:0/2682764653 wait complete. 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.314+0000 7fc396200640 1 Processor -- start 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.314+0000 7fc396200640 1 -- start start 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 0x7fc390199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc390103260 0x7fc3901a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010d200 con 0x7fc390103260 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010d080 con 0x7fc3901016b0 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc396200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010d380 con 0x7fc3901028a0 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:49286/0 (socket says 172.21.15.12:49286) 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 -- 172.21.15.12:0/1026828768 learned_addr learned my addr 172.21.15.12:0/1026828768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc395800640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc390103260 0x7fc3901a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc38fe00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 0x7fc390199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 -- 172.21.15.12:0/1026828768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 msgr2=0x7fc390199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 0x7fc390199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 -- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc390103260 msgr2=0x7fc3901a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc390103260 0x7fc3901a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc394e00640 1 -- 172.21.15.12:0/1026828768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901a06b0 con 0x7fc3901016b0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.315+0000 7fc38fe00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 0x7fc390199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc394e00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc384031500 tx=0x7fc384003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38403f070 con 0x7fc3901016b0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3901a0940 con 0x7fc3901016b0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc384031c50 con 0x7fc3901016b0 2024-04-26T22:02:37.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384035030 con 0x7fc3901016b0 2024-04-26T22:02:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.316+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901a0e80 con 0x7fc3901016b0 2024-04-26T22:02:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.317+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3901a10b0 con 0x7fc3901016b0 2024-04-26T22:02:37.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.319+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fc384042070 con 0x7fc3901016b0 2024-04-26T22:02:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.321+0000 7fc38d600640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 0x7fc3780a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.321+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fc3840f1160 con 0x7fc3901016b0 2024-04-26T22:02:37.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.322+0000 7fc38fe00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 0x7fc3780a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.323+0000 7fc38fe00640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 0x7fc3780a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc390199fa0 tx=0x7fc37c005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: Manager daemon x is now available 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:36 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.329+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc3840bd2d0 con 0x7fc3901016b0 2024-04-26T22:02:37.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: Manager daemon x is now available 2024-04-26T22:02:37.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:37.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T22:02:37.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:36 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T22:02:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.563+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc39006a7a0 con 0x7fc3780a3990 2024-04-26T22:02:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.678+0000 7fc38d600640 1 -- 172.21.15.12:0/1026828768 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc39006a7a0 con 0x7fc3780a3990 2024-04-26T22:02:37.679 INFO:teuthology.orchestra.run.smithi012.stdout:Retention added to path / 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 msgr2=0x7fc3780a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 0x7fc3780a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc390199fa0 tx=0x7fc37c005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 msgr2=0x7fc390198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc384031500 tx=0x7fc384003c10 comp rx=0 tx=0).stop 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 shutdown_connections 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc3780a3990 0x7fc3780a5e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc390103260 0x7fc3901a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3901028a0 0x7fc390199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 --2- 172.21.15.12:0/1026828768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3901016b0 0x7fc390198ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 >> 172.21.15.12:0/1026828768 conn(0x7fc3900fcc20 msgr2=0x7fc39006a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 shutdown_connections 2024-04-26T22:02:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.682+0000 7fc396200640 1 -- 172.21.15.12:0/1026828768 wait complete. 2024-04-26T22:02:37.699 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 -- 172.21.15.12:0/4169880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec1016b0 msgr2=0x7f53ec101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 --2- 172.21.15.12:0/4169880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec1016b0 0x7f53ec101ad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009900 tx=0x7f53dc02a3f0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 -- 172.21.15.12:0/4169880279 shutdown_connections 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 --2- 172.21.15.12:0/4169880279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec103260 0x7f53ec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 --2- 172.21.15.12:0/4169880279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1028a0 0x7f53ec102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 --2- 172.21.15.12:0/4169880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec1016b0 0x7f53ec101ad0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 -- 172.21.15.12:0/4169880279 >> 172.21.15.12:0/4169880279 conn(0x7f53ec0fcc20 msgr2=0x7f53ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 -- 172.21.15.12:0/4169880279 shutdown_connections 2024-04-26T22:02:37.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.882+0000 7f53f1000640 1 -- 172.21.15.12:0/4169880279 wait complete. 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.883+0000 7f53f1000640 1 Processor -- start 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.883+0000 7f53f1000640 1 -- start start 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1016b0 0x7f53ec19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec103260 0x7f53ec1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec10e360 con 0x7f53ec1016b0 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec10e1e0 con 0x7f53ec103260 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53f1000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec10e4e0 con 0x7f53ec1028a0 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1016b0 0x7f53ec19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:53252/0 (socket says 172.21.15.12:53252) 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 -- 172.21.15.12:0/2994516093 learned_addr learned my addr 172.21.15.12:0/2994516093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53ebe00640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec103260 0x7f53ec1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 -- 172.21.15.12:0/2994516093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec103260 msgr2=0x7f53ec1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec103260 0x7f53ec1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 -- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1016b0 msgr2=0x7f53ec19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1016b0 0x7f53ec19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.884+0000 7f53eaa00640 1 -- 172.21.15.12:0/2994516093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1a2890 con 0x7f53ec1028a0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53eaa00640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9f0 tx=0x7f53d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d800cd80 con 0x7f53ec1028a0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d80108e0 con 0x7f53ec1028a0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8018980 con 0x7f53ec1028a0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1a2b80 con 0x7f53ec1028a0 2024-04-26T22:02:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.885+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1a30f0 con 0x7f53ec1028a0 2024-04-26T22:02:37.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.886+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec1a32c0 con 0x7f53ec1028a0 2024-04-26T22:02:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.889+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f53d8010430 con 0x7f53ec1028a0 2024-04-26T22:02:37.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.890+0000 7f53e3e00640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 0x7f53d00a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:37.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.890+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f53d80d0c50 con 0x7f53ec1028a0 2024-04-26T22:02:37.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.890+0000 7f53eb400640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 0x7f53d00a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:37.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.891+0000 7f53eb400640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 0x7f53d00a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005ab0 tx=0x7f53dc005a20 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:37.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:37.891+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f53d809d410 con 0x7f53ec1028a0 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: [26/Apr/2024:22:02:37] ENGINE Bus STARTING 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: mgrmap e21: x(active, since 1.1022s), standbys: y, z 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.129 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: [26/Apr/2024:22:02:37] ENGINE Serving on http://172.21.15.12:8765 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: [26/Apr/2024:22:02:37] ENGINE Serving on https://172.21.15.12:7150 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: [26/Apr/2024:22:02:37] ENGINE Bus STARTED 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:37 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: [26/Apr/2024:22:02:37] ENGINE Bus STARTING 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: mgrmap e21: x(active, since 1.1022s), standbys: y, z 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: [26/Apr/2024:22:02:37] ENGINE Serving on http://172.21.15.12:8765 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: [26/Apr/2024:22:02:37] ENGINE Serving on https://172.21.15.12:7150 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: [26/Apr/2024:22:02:37] ENGINE Bus STARTED 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:37 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.161+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f53ec10bbd0 con 0x7f53d00a3a10 2024-04-26T22:02:38.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.170+0000 7f53e3e00640 1 -- 172.21.15.12:0/2994516093 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f53ec10bbd0 con 0x7f53d00a3a10 2024-04-26T22:02:38.171 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T22:02:37", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T22:02:38.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.172+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 msgr2=0x7f53d00a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.172+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 0x7f53d00a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005ab0 tx=0x7f53dc005a20 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.172+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 msgr2=0x7f53ec19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.172+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9f0 tx=0x7f53d800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 shutdown_connections 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f53d00a3a10 0x7f53d00a5ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53ec103260 0x7f53ec1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53ec1028a0 0x7f53ec19b1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 --2- 172.21.15.12:0/2994516093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec1016b0 0x7f53ec19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 >> 172.21.15.12:0/2994516093 conn(0x7f53ec0fcc20 msgr2=0x7f53ec10a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 shutdown_connections 2024-04-26T22:02:38.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.173+0000 7f53f1000640 1 -- 172.21.15.12:0/2994516093 wait complete. 2024-04-26T22:02:38.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T22:02:38.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: [26/Apr/2024:22:02:37] ENGINE Bus STARTING 2024-04-26T22:02:38.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: mgrmap e21: x(active, since 1.1022s), standbys: y, z 2024-04-26T22:02:38.243 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: [26/Apr/2024:22:02:37] ENGINE Serving on http://172.21.15.12:8765 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: [26/Apr/2024:22:02:37] ENGINE Serving on https://172.21.15.12:7150 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: [26/Apr/2024:22:02:37] ENGINE Bus STARTED 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:37 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 -- 172.21.15.12:0/2946594656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 msgr2=0x7f1bf8109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 --2- 172.21.15.12:0/2946594656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf8109200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009860 tx=0x7f1be802a340 comp rx=0 tx=0).stop 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 -- 172.21.15.12:0/2946594656 shutdown_connections 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 --2- 172.21.15.12:0/2946594656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 0x7f1bf8111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 --2- 172.21.15.12:0/2946594656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 0x7f1bf810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 --2- 172.21.15.12:0/2946594656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf8109200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 -- 172.21.15.12:0/2946594656 >> 172.21.15.12:0/2946594656 conn(0x7f1bf8077dc0 msgr2=0x7f1bf807a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 -- 172.21.15.12:0/2946594656 shutdown_connections 2024-04-26T22:02:38.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.375+0000 7f1bfca00640 1 -- 172.21.15.12:0/2946594656 wait complete. 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 Processor -- start 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 -- start start 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 0x7f1bf819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 0x7f1bf81a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81148e0 con 0x7f1bf8109fd0 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf8114760 con 0x7f1bf810a990 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.376+0000 7f1bfca00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf8114a60 con 0x7f1bf8108de0 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:53296/0 (socket says 172.21.15.12:53296) 2024-04-26T22:02:38.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 -- 172.21.15.12:0/3012107403 learned_addr learned my addr 172.21.15.12:0/3012107403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bede00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 0x7f1bf819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf7200640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 0x7f1bf81a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 -- 172.21.15.12:0/3012107403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 msgr2=0x7f1bf81a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 0x7f1bf81a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 -- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 msgr2=0x7f1bf819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 0x7f1bf819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 -- 172.21.15.12:0/3012107403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf81a6c00 con 0x7f1bf8108de0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.377+0000 7f1bf6800640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1be80057c0 tx=0x7f1be8032eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.378+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8037040 con 0x7f1bf8108de0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.378+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be8030cd0 con 0x7f1bf8108de0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.378+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8005310 con 0x7f1bf8108de0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.378+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf81a6e90 con 0x7f1bf8108de0 2024-04-26T22:02:38.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.378+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf81a7240 con 0x7f1bf8108de0 2024-04-26T22:02:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.379+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf8110210 con 0x7f1bf8108de0 2024-04-26T22:02:38.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.381+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f1be8043020 con 0x7f1bf8108de0 2024-04-26T22:02:38.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.382+0000 7f1befe00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 0x7f1bdc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:38.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.382+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f1be80f01b0 con 0x7f1bf8108de0 2024-04-26T22:02:38.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.382+0000 7f1bede00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 0x7f1bdc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:38.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.383+0000 7f1bede00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 0x7f1bdc0a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bf81a04f0 tx=0x7f1be0009260 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:38.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.386+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1be80bc550 con 0x7f1bf8108de0 2024-04-26T22:02:38.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.619+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1bf8062d70 con 0x7f1bdc0a3d90 2024-04-26T22:02:38.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.630+0000 7f1befe00640 1 -- 172.21.15.12:0/3012107403 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1bf8062d70 con 0x7f1bdc0a3d90 2024-04-26T22:02:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 msgr2=0x7f1bdc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 0x7f1bdc0a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bf81a04f0 tx=0x7f1be0009260 comp rx=0 tx=0).stop 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 msgr2=0x7f1bf819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1be80057c0 tx=0x7f1be8032eb0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 shutdown_connections 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1bdc0a3d90 0x7f1bdc0a6250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bf810a990 0x7f1bf81a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1bf8109fd0 0x7f1bf819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 --2- 172.21.15.12:0/3012107403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bf8108de0 0x7f1bf819efe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 >> 172.21.15.12:0/3012107403 conn(0x7f1bf8077dc0 msgr2=0x7f1bf810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 shutdown_connections 2024-04-26T22:02:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:02:38.634+0000 7f1bfca00640 1 -- 172.21.15.12:0/3012107403 wait complete. 2024-04-26T22:02:38.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T22:02:38.735 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T22:02:38.747 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T22:02:38.764 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:02:38.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T22:02:38.769 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T22:02:38.783 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:02:38.783 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T22:02:38.783 INFO:tasks.workunit:timeout=3h 2024-04-26T22:02:38.783 INFO:tasks.workunit:cleanup=True 2024-04-26T22:02:38.784 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T22:02:38.830 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 22:00:12.324773274 +0000 2024-04-26T22:02:38.831 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 22:00:12.324773274 +0000 2024-04-26T22:02:38.831 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 22:02:34.042763128 +0000 2024-04-26T22:02:38.831 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-26 22:00:12.324773274 +0000 2024-04-26T22:02:38.831 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T22:02:38.831 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T22:02:38.952 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:02:39.003 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T22:02:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='client.25385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-26T22:02:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:39 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='client.25385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:39 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='client.25385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='client.25387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:39.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:39 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:40 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:40 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:02:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:40.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:40 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:41 smithi154 ceph-mon[32578]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:41 smithi077 ceph-mon[32280]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:41.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:41 smithi012 ceph-mon[28184]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-04-26T22:02:43.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:43.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:43 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.721 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:43.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:43 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:02:43.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:43.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:43 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:44.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:44 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:44 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:44.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:44 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:45.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:45 smithi154 ceph-mon[32578]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:45 smithi154 ceph-mon[32578]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:45 smithi154 ceph-mon[32578]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 74 op/s 2024-04-26T22:02:45.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:45 smithi154 ceph-mon[32578]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:45 smithi077 ceph-mon[32280]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:45 smithi077 ceph-mon[32280]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:45 smithi077 ceph-mon[32280]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 74 op/s 2024-04-26T22:02:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:45 smithi077 ceph-mon[32280]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:45.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:45 smithi012 ceph-mon[28184]: Updating smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:45 smithi012 ceph-mon[28184]: Updating smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:45.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:45 smithi012 ceph-mon[28184]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 74 op/s 2024-04-26T22:02:45.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:45 smithi012 ceph-mon[28184]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-26T22:02:46.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:46.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:46.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:46 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:46.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:46 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: Updating smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:46.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:46 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:02:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:47 smithi154 ceph-mon[32578]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-04-26T22:02:47.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:47 smithi077 ceph-mon[32280]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-04-26T22:02:47.749 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:47 smithi012 ceph-mon[28184]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-04-26T22:02:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:49 smithi154 ceph-mon[32578]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 85 KiB/s wr, 48 op/s 2024-04-26T22:02:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:49 smithi077 ceph-mon[32280]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 85 KiB/s wr, 48 op/s 2024-04-26T22:02:49.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:49 smithi012 ceph-mon[28184]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 85 KiB/s wr, 48 op/s 2024-04-26T22:02:51.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:51 smithi154 ceph-mon[32578]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:51 smithi077 ceph-mon[32280]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:51 smithi012 ceph-mon[28184]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.012+0000 7f5dc0a00640 1 -- 172.21.15.12:0/3303973441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 msgr2=0x7f5dbc10ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.012+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/3303973441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc10ece0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009860 tx=0x7f5da402a3d0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 -- 172.21.15.12:0/3303973441 shutdown_connections 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/3303973441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 0x7f5dbc111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/3303973441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc10ece0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/3303973441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 0x7f5dbc075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 -- 172.21.15.12:0/3303973441 >> 172.21.15.12:0/3303973441 conn(0x7f5dbc079dc0 msgr2=0x7f5dbc07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 -- 172.21.15.12:0/3303973441 shutdown_connections 2024-04-26T22:02:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.013+0000 7f5dc0a00640 1 -- 172.21.15.12:0/3303973441 wait complete. 2024-04-26T22:02:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 Processor -- start 2024-04-26T22:02:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 -- start start 2024-04-26T22:02:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 0x7f5dbc19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 0x7f5dbc19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc114e50 con 0x7f5dbc10f220 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc114cd0 con 0x7f5dbc075550 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.014+0000 7f5dc0a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc114fd0 con 0x7f5dbc074c30 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 0x7f5dbc19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:41908/0 (socket says 172.21.15.12:41908) 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 -- 172.21.15.12:0/1883355349 learned_addr learned my addr 172.21.15.12:0/1883355349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5dba800640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 0x7f5dbc19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 -- 172.21.15.12:0/1883355349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 msgr2=0x7f5dbc19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 0x7f5dbc19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 -- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 msgr2=0x7f5dbc19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 0x7f5dbc19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.015+0000 7f5db3e00640 1 -- 172.21.15.12:0/1883355349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1a0220 con 0x7f5dbc075550 2024-04-26T22:02:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009980 tx=0x7f5da4030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5db2a00640 1 -- 172.21.15.12:0/1883355349 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4040070 con 0x7f5dbc075550 2024-04-26T22:02:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5db2a00640 1 -- 172.21.15.12:0/1883355349 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da4005570 con 0x7f5dbc075550 2024-04-26T22:02:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5db2a00640 1 -- 172.21.15.12:0/1883355349 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da40046c0 con 0x7f5dbc075550 2024-04-26T22:02:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1a04b0 con 0x7f5dbc075550 2024-04-26T22:02:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.016+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a7d50 con 0x7f5dbc075550 2024-04-26T22:02:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.019+0000 7f5dc381b740 1 Processor -- start 2024-04-26T22:02:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5db2a00640 1 -- 172.21.15.12:0/1883355349 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 138822+0+0 (secure 0 0 0) 0x7f5da4043070 con 0x7f5dbc075550 2024-04-26T22:02:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 -- start start 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 0x55665b8ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55665b803380 con 0x55665b89ddd0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55665b79fc30 con 0x55665b8a8fb0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dc381b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55665b7ff340 con 0x55665b89e6f0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5dba800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.020+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 0x55665b8ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54978/0 (socket says 172.21.15.12:54978) 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 -- 172.21.15.12:0/1791720699 learned_addr learned my addr 172.21.15.12:0/1791720699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 -- 172.21.15.12:0/1791720699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 msgr2=0x55665b8a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8a8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 -- 172.21.15.12:0/1791720699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 msgr2=0x55665b8ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 0x55665b8ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 -- 172.21.15.12:0/1791720699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b7c1940 con 0x55665b89ddd0 2024-04-26T22:02:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 0x55665b8ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5dba800640 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e480 tx=0x7f5db40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ce1c293200ca3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.021+0000 7f5db2a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 0x7f5da00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.022+0000 7f5db2a00640 1 -- 172.21.15.12:0/1883355349 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5da40f1820 con 0x7f5dbc075550 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.022+0000 7f5dab000640 1 -- 172.21.15.12:0/1791720699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4003970 con 0x55665b89ddd0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.022+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 0x7f5da00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.022+0000 7f5dab000640 1 -- 172.21.15.12:0/1791720699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db400d7d0 con 0x55665b89ddd0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.022+0000 7f5dab000640 1 -- 172.21.15.12:0/1791720699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004350 con 0x55665b89ddd0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 -- 172.21.15.12:0/1791720699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 msgr2=0x55665b89e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 0x7f5da00a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55665b89f370 tx=0x7f5dac005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e480 tx=0x7f5db40095c0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 -- 172.21.15.12:0/1791720699 shutdown_connections 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b8a8fb0 0x55665b8ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1791720699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b89e1b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 -- 172.21.15.12:0/1791720699 >> 172.21.15.12:0/1791720699 conn(0x55665b89a2e0 msgr2=0x55665b89a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 -- 172.21.15.12:0/1791720699 shutdown_connections 2024-04-26T22:02:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.023+0000 7f5dc381b740 1 -- 172.21.15.12:0/1791720699 wait complete. 2024-04-26T22:02:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.024+0000 7f5dc381b740 1 -- start start 2024-04-26T22:02:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.024+0000 7f5dc381b740 1 Processor -- start 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89e6f0 0x55665b8a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55665b8abbe0 con 0x55665b89e6f0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55665b78ac60 con 0x55665b89ddd0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55665b8abd60 con 0x55665b8a8fb0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dc381b740 10 client.0 _ll_register_callbacks cb 0x55665b7b6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89e6f0 0x55665b8a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:41938/0 (socket says 172.21.15.12:41938) 2024-04-26T22:02:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 -- 172.21.15.12:0/280278917 learned_addr learned my addr 172.21.15.12:0/280278917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5dba800640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 -- 172.21.15.12:0/280278917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 msgr2=0x55665b8a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 -- 172.21.15.12:0/280278917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89e6f0 msgr2=0x55665b8a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89e6f0 0x55665b8a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.025+0000 7f5db9e00640 1 -- 172.21.15.12:0/280278917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665b799510 con 0x55665b89ddd0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5db9e00640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0045e0 tx=0x7f5dac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11a55f84b9e1840 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5d9f400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac01e680 con 0x55665b89ddd0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db400d7d0 con 0x55665b89ddd0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5dac01e820 con 0x55665b89ddd0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5dac01ea20 con 0x55665b89ddd0 2024-04-26T22:02:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5dc381b740 10 client.25415 fetch_fsmap learned FSMap version 21 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.026+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55665b515210 con 0x55665b89ddd0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db400d7d0 con 0x55665b89ddd0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5dac027070 con 0x55665b89ddd0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 10 client.25415 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 20 client.25415 populate_metadata read hostname 'smithi012' 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b8b8db0 0x55665b8bb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 4 client.25415 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55665b810f90 con 0x55665b8b8db0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.027+0000 7f5db3e00640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b8b8db0 0x55665b8bb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.028+0000 7f5db3e00640 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b8b8db0 0x55665b8bb1e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dadb93039fc1c16 server_cookie=5ed907c979b9c4a2 in_seq=0 out_seq=0 2024-04-26T22:02:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.028+0000 7f5d9f400640 10 client.25415 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5d9f400640 1 -- 172.21.15.12:0/280278917 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55665b810f90 con 0x55665b8b8db0 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5d9f400640 10 client.25415 handle_command_reply: tid=0 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 client.25415 shutdown 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 msgr2=0x55665b8a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0045e0 tx=0x7f5dac004730 comp rx=0 tx=0).stop 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 shutdown_connections 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b8b8db0 0x55665b8bb1e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89e6f0 0x55665b8a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5d9f400640 0 client.25415 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 --2- 172.21.15.12:0/280278917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8a5f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 >> 172.21.15.12:0/280278917 conn(0x55665b89a2e0 msgr2=0x55665b8a0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 shutdown_connections 2024-04-26T22:02:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 1 -- 172.21.15.12:0/280278917 wait complete. 2024-04-26T22:02:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.029+0000 7f5dc381b740 20 client.25415 trim_cache size 0 max 16384 2024-04-26T22:02:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 Processor -- start 2024-04-26T22:02:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 -- start start 2024-04-26T22:02:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1afe40 con 0x55665b89ddd0 2024-04-26T22:02:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b26c0 con 0x55665b89e6f0 2024-04-26T22:02:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.046+0000 7f5dc381b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b04b0 con 0x55665b8a8fb0 2024-04-26T22:02:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:41942/0 (socket says 172.21.15.12:41942) 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db3e00640 1 -- 172.21.15.12:0/3264303002 learned_addr learned my addr 172.21.15.12:0/3264303002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54996/0 (socket says 172.21.15.12:54996) 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 -- 172.21.15.12:0/3264303002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 msgr2=0x55665b8a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 -- 172.21.15.12:0/3264303002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 msgr2=0x55665b8a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db9e00640 1 -- 172.21.15.12:0/3264303002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665b5e1620 con 0x55665b89ddd0 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.047+0000 7f5db3e00640 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.048+0000 7f5db9e00640 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5dac029f80 tx=0x7f5dac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e402b9bb4dd81d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.048+0000 7f5daa600640 1 -- 172.21.15.12:0/3264303002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac01e680 con 0x55665b89ddd0 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.048+0000 7f5daa600640 1 -- 172.21.15.12:0/3264303002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac01e820 con 0x55665b89ddd0 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.048+0000 7f5daa600640 1 -- 172.21.15.12:0/3264303002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac004a90 con 0x55665b89ddd0 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- 172.21.15.12:0/3264303002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 msgr2=0x55665b8b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5dac029f80 tx=0x7f5dac009b60 comp rx=0 tx=0).stop 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- 172.21.15.12:0/3264303002 shutdown_connections 2024-04-26T22:02:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b8a8fb0 0x55665b8a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89e6f0 0x55665b8a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 --2- 172.21.15.12:0/3264303002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b89ddd0 0x55665b8b4aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- 172.21.15.12:0/3264303002 >> 172.21.15.12:0/3264303002 conn(0x55665b89a2e0 msgr2=0x55665b8a7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- 172.21.15.12:0/3264303002 shutdown_connections 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- 172.21.15.12:0/3264303002 wait complete. 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 -- start start 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.049+0000 7f5dc381b740 1 Processor -- start 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b0b80 con 0x55665b8a8fb0 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b1ff0 con 0x55665b89ddd0 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b1250 con 0x55665b89e6f0 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dc381b740 10 client.0 _ll_register_callbacks cb 0x7f5dac009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dba800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5db9e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dba800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55006/0 (socket says 172.21.15.12:55006) 2024-04-26T22:02:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.050+0000 7f5dba800640 1 -- 172.21.15.12:0/1124382960 learned_addr learned my addr 172.21.15.12:0/1124382960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:02:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 -- 172.21.15.12:0/1124382960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 msgr2=0x55665b8ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 -- 172.21.15.12:0/1124382960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 msgr2=0x55665b8acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 -- 172.21.15.12:0/1124382960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665b7bdcb0 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5dba800640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5db9e00640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8b13d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dac029eb0 tx=0x7f5dac005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebed9db6ca8a9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5d9f400640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac023040 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.051+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac01e820 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5dac0055d0 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5dac003e50 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 10 client.25421 fetch_fsmap learned FSMap version 21 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55665b91f220 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac01e820 con 0x55665b89ddd0 2024-04-26T22:02:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5dac027040 con 0x55665b89ddd0 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 10 client.25421 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 20 client.25421 populate_metadata read hostname 'smithi012' 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b91f6a0 0x55665b921a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 4 client.25421 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.052+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55665b80ae00 con 0x55665b91f6a0 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.053+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b91f6a0 0x55665b921a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.053+0000 7f5db3e00640 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b91f6a0 0x55665b921a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8083bda8102419ee server_cookie=d44110aaa659bcb in_seq=0 out_seq=0 2024-04-26T22:02:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.053+0000 7f5d9f400640 10 client.25421 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5d9f400640 1 -- 172.21.15.12:0/1124382960 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55665b80ae00 con 0x55665b91f6a0 2024-04-26T22:02:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5d9f400640 10 client.25421 handle_command_reply: tid=0 2024-04-26T22:02:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 client.25421 shutdown 2024-04-26T22:02:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 msgr2=0x55665b8b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8b13d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dac029eb0 tx=0x7f5dac005250 comp rx=0 tx=0).stop 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 shutdown_connections 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55665b91f6a0 0x55665b921a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55665b8a8fb0 0x55665b8acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55665b89e6f0 0x55665b8ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 --2- 172.21.15.12:0/1124382960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55665b89ddd0 0x55665b8b13d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5d9f400640 0 client.25421 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 >> 172.21.15.12:0/1124382960 conn(0x55665b89a2e0 msgr2=0x55665b899440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 shutdown_connections 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 1 -- 172.21.15.12:0/1124382960 wait complete. 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.068+0000 7f5dc381b740 20 client.25421 trim_cache size 0 max 16384 2024-04-26T22:02:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:02:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:02:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:02:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 msgr2=0x7f5da00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 0x7f5da00a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55665b89f370 tx=0x7f5dac005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 msgr2=0x7f5dbc19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009980 tx=0x7f5da4030fa0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 shutdown_connections 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5da00a3e60 0x7f5da00a6320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5dbc10f220 0x7f5dbc19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dbc075550 0x7f5dbc19b770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 --2- 172.21.15.12:0/1883355349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dbc074c30 0x7f5dbc19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.069+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 >> 172.21.15.12:0/1883355349 conn(0x7f5dbc079dc0 msgr2=0x7f5dbc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.070+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 shutdown_connections 2024-04-26T22:02:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:02:52.070+0000 7f5dc0a00640 1 -- 172.21.15.12:0/1883355349 wait complete. 2024-04-26T22:02:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:53 smithi154 ceph-mon[32578]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:53 smithi077 ceph-mon[32280]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:53.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:53 smithi012 ceph-mon[28184]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-26T22:02:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:55 smithi154 ceph-mon[32578]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-26T22:02:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:55 smithi077 ceph-mon[32280]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-26T22:02:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:55 smithi012 ceph-mon[28184]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-26T22:02:57.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:57 smithi154 ceph-mon[32578]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:02:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:57 smithi077 ceph-mon[32280]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:02:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:57 smithi012 ceph-mon[28184]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:02:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:02:59 smithi154 ceph-mon[32578]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T22:02:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:02:59 smithi077 ceph-mon[32280]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T22:02:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:02:59 smithi012 ceph-mon[28184]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T22:03:01.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:03:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:03:00.704+0000 7fddb4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:03:01.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:03:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:03:00.704+0000 7fddb4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:03:01.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:03:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:03:00.704+0000 7fddb4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:03:01.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:03:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:03:00.704+0000 7fddb4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:03:01.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:03:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:03:00.704+0000 7fddb4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:03:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:01 smithi077 ceph-mon[32280]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:01 smithi012 ceph-mon[28184]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:01 smithi154 ceph-mon[32578]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:03 smithi077 ceph-mon[32280]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:03 smithi077 ceph-mon[32280]: mgrmap e24: x(active, since 26s), standbys: y, z 2024-04-26T22:03:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:03 smithi012 ceph-mon[28184]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:03 smithi012 ceph-mon[28184]: mgrmap e24: x(active, since 26s), standbys: y, z 2024-04-26T22:03:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:03 smithi154 ceph-mon[32578]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T22:03:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:03 smithi154 ceph-mon[32578]: mgrmap e24: x(active, since 26s), standbys: y, z 2024-04-26T22:03:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 -- 172.21.15.12:0/3698261311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8108750 msgr2=0x7fe2c8108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/3698261311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8108750 0x7fe2c8108b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009890 tx=0x7fe2b802a360 comp rx=0 tx=0).stop 2024-04-26T22:03:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 -- 172.21.15.12:0/3698261311 shutdown_connections 2024-04-26T22:03:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/3698261311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8103100 0x7fe2c810f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/3698261311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8102740 0x7fe2c8102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/3698261311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8108750 0x7fe2c8108b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 -- 172.21.15.12:0/3698261311 >> 172.21.15.12:0/3698261311 conn(0x7fe2c80fe270 msgr2=0x7fe2c8100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 -- 172.21.15.12:0/3698261311 shutdown_connections 2024-04-26T22:03:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.218+0000 7fe2c7e00640 1 -- 172.21.15.12:0/3698261311 wait complete. 2024-04-26T22:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.219+0000 7fe2c7e00640 1 Processor -- start 2024-04-26T22:03:05.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.219+0000 7fe2c7e00640 1 -- start start 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 0x7fe2c8198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 0x7fe2c81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8108750 0x7fe2c819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112a10 con 0x7fe2c8102740 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112890 con 0x7fe2c8103100 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8112b90 con 0x7fe2c8108750 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 0x7fe2c8198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 0x7fe2c81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 0x7fe2c8198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48710/0 (socket says 172.21.15.12:48710) 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6a00640 1 -- 172.21.15.12:0/842494589 learned_addr learned my addr 172.21.15.12:0/842494589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c7400640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8108750 0x7fe2c819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 -- 172.21.15.12:0/842494589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8108750 msgr2=0x7fe2c819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8108750 0x7fe2c819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 -- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 msgr2=0x7fe2c8198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 0x7fe2c8198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.220+0000 7fe2c6000640 1 -- 172.21.15.12:0/842494589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c819dfa0 con 0x7fe2c8103100 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2c6000640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 0x7fe2c81994f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9f0 tx=0x7fe2b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2bf400640 1 -- 172.21.15.12:0/842494589 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400cd80 con 0x7fe2c8103100 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c819e290 con 0x7fe2c8103100 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2bf400640 1 -- 172.21.15.12:0/842494589 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b40035f0 con 0x7fe2c8103100 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2bf400640 1 -- 172.21.15.12:0/842494589 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4010690 con 0x7fe2c8103100 2024-04-26T22:03:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.221+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81a5b20 con 0x7fe2c8103100 2024-04-26T22:03:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 Processor -- start 2024-04-26T22:03:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 -- start start 2024-04-26T22:03:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e0c5d0 0x557465e0e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557465d62380 con 0x557465e013f0 2024-04-26T22:03:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557465cfec30 con 0x557465e01d10 2024-04-26T22:03:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2d0369740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557465d5e340 con 0x557465e0c5d0 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.225+0000 7fe2bf400640 1 -- 172.21.15.12:0/842494589 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fe2b40037a0 con 0x7fe2c8103100 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48712/0 (socket says 172.21.15.12:48712) 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 -- 172.21.15.12:0/1936170717 learned_addr learned my addr 172.21.15.12:0/1936170717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c6000640 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e0c5d0 0x557465e0e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 -- 172.21.15.12:0/1936170717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e0c5d0 msgr2=0x557465e0e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e0c5d0 0x557465e0e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 -- 172.21.15.12:0/1936170717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 msgr2=0x557465e0c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e0c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 -- 172.21.15.12:0/1936170717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557465d20940 con 0x557465e013f0 2024-04-26T22:03:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.226+0000 7fe2c7400640 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000e480 tx=0x7fe2c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cf21da8c3666ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2b3400640 1 -- 172.21.15.12:0/1936170717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0003970 con 0x557465e013f0 2024-04-26T22:03:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2b3400640 1 -- 172.21.15.12:0/1936170717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c000d7d0 con 0x557465e013f0 2024-04-26T22:03:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2b3400640 1 -- 172.21.15.12:0/1936170717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00043d0 con 0x557465e013f0 2024-04-26T22:03:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2bf400640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 0x7fe2ac0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2d0369740 1 -- 172.21.15.12:0/1936170717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 msgr2=0x557465e017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2d0369740 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000e480 tx=0x7fe2c00095c0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.227+0000 7fe2bf400640 1 -- 172.21.15.12:0/842494589 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fe2b40d1180 con 0x7fe2c8103100 2024-04-26T22:03:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 0x7fe2ac0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 -- 172.21.15.12:0/1936170717 shutdown_connections 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e0c5d0 0x557465e0e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e0c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 --2- 172.21.15.12:0/1936170717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e017d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 -- 172.21.15.12:0/1936170717 >> 172.21.15.12:0/1936170717 conn(0x557465dfd900 msgr2=0x557465dfdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 -- 172.21.15.12:0/1936170717 shutdown_connections 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 -- 172.21.15.12:0/1936170717 wait complete. 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 0x7fe2ac0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009230 tx=0x7fe2b802c040 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.228+0000 7fe2d0369740 1 -- start start 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 Processor -- start 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e013f0 0x557465e09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e09ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557465e0f200 con 0x557465e0c5d0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557465ce9c60 con 0x557465e01d10 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557465e0f380 con 0x557465e013f0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2d0369740 10 client.0 _ll_register_callbacks cb 0x557465d15e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48720/0 (socket says 172.21.15.12:48720) 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.229+0000 7fe2c7400640 1 -- 172.21.15.12:0/282855717 learned_addr learned my addr 172.21.15.12:0/282855717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e013f0 0x557465e09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c6000640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e09ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 -- 172.21.15.12:0/282855717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e013f0 msgr2=0x557465e09570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e013f0 0x557465e09570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 -- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 msgr2=0x557465e09ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e09ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 -- 172.21.15.12:0/282855717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557465cf8510 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2c7400640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005e30 tx=0x7fe2c0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbdb03e4a07acf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.230+0000 7fe2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0027020 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fe2c0004c50 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c000d7d0 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2c0032050 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2d0369740 10 client.15312 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557465a74210 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c000d7d0 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe2c000b040 con 0x557465e0c5d0 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2d0369740 10 client.15312 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.231+0000 7fe2d0369740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.232+0000 7fe2d0369740 20 client.15312 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.232+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e1c2c0 0x557465e1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.232+0000 7fe2d0369740 4 client.15312 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.232+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557465d6ff90 con 0x557465e1c2c0 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.232+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e1c2c0 0x557465e1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.233+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e1c2c0 0x557465e1e6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7155e2edb63162b server_cookie=a22bda6fd176e644 in_seq=0 out_seq=0 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.233+0000 7fe2a7e00640 10 client.15312 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.233+0000 7fe2a7e00640 1 -- 172.21.15.12:0/282855717 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557465d6ff90 con 0x557465e1c2c0 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.233+0000 7fe2a7e00640 10 client.15312 handle_command_reply: tid=0 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.233+0000 7fe2d0369740 1 client.15312 shutdown 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 msgr2=0x557465e09ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005e30 tx=0x7fe2c0003c90 comp rx=0 tx=0).stop 2024-04-26T22:03:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 shutdown_connections 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e1c2c0 0x557465e1e6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e0c5d0 0x557465e09ff0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2a7e00640 0 client.15312 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e01d10 0x557465e09ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 --2- 172.21.15.12:0/282855717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e013f0 0x557465e09570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 >> 172.21.15.12:0/282855717 conn(0x557465dfd900 msgr2=0x557465df8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 shutdown_connections 2024-04-26T22:03:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 1 -- 172.21.15.12:0/282855717 wait complete. 2024-04-26T22:03:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.234+0000 7fe2d0369740 20 client.15312 trim_cache size 0 max 16384 2024-04-26T22:03:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.250+0000 7fe2d0369740 1 Processor -- start 2024-04-26T22:03:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 -- start start 2024-04-26T22:03:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e08e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ab9a0 con 0x557465e013f0 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ae220 con 0x557465e0c5d0 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2d0369740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ac010 con 0x557465e01d10 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48736/0 (socket says 172.21.15.12:48736) 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2c6a00640 1 -- 172.21.15.12:0/811843820 learned_addr learned my addr 172.21.15.12:0/811843820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2c7400640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.251+0000 7fe2c6000640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e08e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 -- 172.21.15.12:0/811843820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 msgr2=0x557465e0ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e0ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 -- 172.21.15.12:0/811843820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 msgr2=0x557465e180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 -- 172.21.15.12:0/811843820 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557465e0bb40 con 0x557465e01d10 2024-04-26T22:03:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c7400640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2c6000640 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e08e70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ce340 tx=0x7fe2b40bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c3c48070248c5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2b2a00640 1 -- 172.21.15.12:0/811843820 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b409cac0 con 0x557465e01d10 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2b2a00640 1 -- 172.21.15.12:0/811843820 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b40a0e70 con 0x557465e01d10 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.252+0000 7fe2b2a00640 1 -- 172.21.15.12:0/811843820 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40d30f0 con 0x557465e01d10 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- 172.21.15.12:0/811843820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 msgr2=0x557465e08e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e08e70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ce340 tx=0x7fe2b40bd040 comp rx=0 tx=0).stop 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- 172.21.15.12:0/811843820 shutdown_connections 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e08e70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 --2- 172.21.15.12:0/811843820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- 172.21.15.12:0/811843820 >> 172.21.15.12:0/811843820 conn(0x557465dfd900 msgr2=0x557465df8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- 172.21.15.12:0/811843820 shutdown_connections 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- 172.21.15.12:0/811843820 wait complete. 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 -- start start 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.253+0000 7fe2d0369740 1 Processor -- start 2024-04-26T22:03:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e06560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81ac6e0 con 0x557465e013f0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81adb50 con 0x557465e0c5d0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81acdb0 con 0x557465e01d10 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2d0369740 10 client.0 _ll_register_callbacks cb 0x7fe2b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e06560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2c6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2c6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:59128/0 (socket says 172.21.15.12:59128) 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2c6000640 1 -- 172.21.15.12:0/3590455053 learned_addr learned my addr 172.21.15.12:0/3590455053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.254+0000 7fe2c7400640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 -- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 msgr2=0x557465e814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 -- 172.21.15.12:0/3590455053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 msgr2=0x557465e06560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e06560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 -- 172.21.15.12:0/3590455053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557465b40620 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2c6000640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ce3a0 tx=0x7fe2b40d1400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cca5bb27f6b8a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4003ac0 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b40a0e70 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fe2b4009310 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2b4099130 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2d0369740 10 client.25438 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.255+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557465d1ccb0 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b40a0e70 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe2b409bf10 con 0x557465e01d10 2024-04-26T22:03:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 10 client.25438 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 20 client.25438 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e82ed0 0x557465e852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 4 client.25438 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557465d69e00 con 0x557465e82ed0 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.256+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e82ed0 0x557465e852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.257+0000 7fe2c6a00640 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e82ed0 0x557465e852f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6863e774cc53387f server_cookie=210bdf6e20c484fe in_seq=0 out_seq=0 2024-04-26T22:03:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.257+0000 7fe2a7e00640 10 client.25438 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2a7e00640 1 -- 172.21.15.12:0/3590455053 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557465d69e00 con 0x557465e82ed0 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2a7e00640 10 client.25438 handle_command_reply: tid=0 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 client.25438 shutdown 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 msgr2=0x557465e06aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ce3a0 tx=0x7fe2b40d1400 comp rx=0 tx=0).stop 2024-04-26T22:03:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 shutdown_connections 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x557465e82ed0 0x557465e852f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557465e0c5d0 0x557465e814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557465e01d10 0x557465e06aa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 --2- 172.21.15.12:0/3590455053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557465e013f0 0x557465e06560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 >> 172.21.15.12:0/3590455053 conn(0x557465dfd900 msgr2=0x557465e09da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2a7e00640 0 client.25438 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 shutdown_connections 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 1 -- 172.21.15.12:0/3590455053 wait complete. 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.265+0000 7fe2d0369740 20 client.25438 trim_cache size 0 max 16384 2024-04-26T22:03:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 msgr2=0x7fe2ac0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 0x7fe2ac0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009230 tx=0x7fe2b802c040 comp rx=0 tx=0).stop 2024-04-26T22:03:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 msgr2=0x7fe2c81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 0x7fe2c81994f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9f0 tx=0x7fe2b400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 shutdown_connections 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fe2ac0a3e30 0x7fe2ac0a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2c8108750 0x7fe2c819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c8103100 0x7fe2c81994f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 --2- 172.21.15.12:0/842494589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2c8102740 0x7fe2c8198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.266+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 >> 172.21.15.12:0/842494589 conn(0x7fe2c80fe270 msgr2=0x7fe2c80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.267+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 shutdown_connections 2024-04-26T22:03:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:05.267+0000 7fe2c7e00640 1 -- 172.21.15.12:0/842494589 wait complete. 2024-04-26T22:03:05.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:05 smithi077 ceph-mon[32280]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:03:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:05 smithi012 ceph-mon[28184]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:03:05.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:05 smithi154 ceph-mon[32578]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T22:03:07.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:07 smithi077 ceph-mon[32280]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T22:03:07.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:07 smithi012 ceph-mon[28184]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T22:03:07.902 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:07 smithi154 ceph-mon[32578]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T22:03:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:09 smithi077 ceph-mon[32280]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-26T22:03:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:09 smithi012 ceph-mon[28184]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-26T22:03:09.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:09 smithi154 ceph-mon[32578]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-26T22:03:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:11 smithi077 ceph-mon[32280]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:11 smithi012 ceph-mon[28184]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:11.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:11 smithi154 ceph-mon[32578]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:13 smithi077 ceph-mon[32280]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:13 smithi012 ceph-mon[28184]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:13.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:13 smithi154 ceph-mon[32578]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T22:03:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:15 smithi077 ceph-mon[32280]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T22:03:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:15 smithi012 ceph-mon[28184]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T22:03:15.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:15 smithi154 ceph-mon[32578]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T22:03:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:17 smithi077 ceph-mon[32280]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-26T22:03:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:17 smithi012 ceph-mon[28184]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-26T22:03:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:17 smithi154 ceph-mon[32578]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-26T22:03:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 -- 172.21.15.12:0/3215677604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69301038f0 msgr2=0x7f6930103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 --2- 172.21.15.12:0/3215677604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69301038f0 0x7f6930103cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6920009be0 tx=0x7f692002a750 comp rx=0 tx=0).stop 2024-04-26T22:03:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 -- 172.21.15.12:0/3215677604 shutdown_connections 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 --2- 172.21.15.12:0/3215677604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f693010cd10 0x7f693010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 --2- 172.21.15.12:0/3215677604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6930104210 0x7f693010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 --2- 172.21.15.12:0/3215677604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69301038f0 0x7f6930103cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 -- 172.21.15.12:0/3215677604 >> 172.21.15.12:0/3215677604 conn(0x7f69300fd150 msgr2=0x7f69300ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 -- 172.21.15.12:0/3215677604 shutdown_connections 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.411+0000 7f692fe00640 1 -- 172.21.15.12:0/3215677604 wait complete. 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.413+0000 7f692fe00640 1 Processor -- start 2024-04-26T22:03:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.413+0000 7f692fe00640 1 -- start start 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.413+0000 7f692fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 0x7f693019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 0x7f693019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 0x7f693019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69301129c0 con 0x7f69301038f0 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692fe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6930112840 con 0x7f693010cd10 2024-04-26T22:03:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6930112b40 con 0x7f6930104210 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 0x7f693019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 0x7f693019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 0x7f693019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33186/0 (socket says 172.21.15.12:33186) 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692ea00640 1 -- 172.21.15.12:0/1636152369 learned_addr learned my addr 172.21.15.12:0/1636152369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692f400640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 0x7f693019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 -- 172.21.15.12:0/1636152369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 msgr2=0x7f693019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 0x7f693019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 -- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 msgr2=0x7f693019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 0x7f693019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692e000640 1 -- 172.21.15.12:0/1636152369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69301a01d0 con 0x7f6930104210 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.414+0000 7f692f400640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 0x7f693019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f692e000640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 0x7f693019b720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f691800e7a0 tx=0x7f691800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f6927400640 1 -- 172.21.15.12:0/1636152369 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691800ccc0 con 0x7f6930104210 2024-04-26T22:03:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f6927400640 1 -- 172.21.15.12:0/1636152369 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6918009530 con 0x7f6930104210 2024-04-26T22:03:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69301a04c0 con 0x7f6930104210 2024-04-26T22:03:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f6927400640 1 -- 172.21.15.12:0/1636152369 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918018710 con 0x7f6930104210 2024-04-26T22:03:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.415+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69301a7d50 con 0x7f6930104210 2024-04-26T22:03:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.418+0000 7f6927400640 1 -- 172.21.15.12:0/1636152369 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f6918010a00 con 0x7f6930104210 2024-04-26T22:03:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.418+0000 7f6938414740 1 Processor -- start 2024-04-26T22:03:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 -- start start 2024-04-26T22:03:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 0x55f8577bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f857721380 con 0x55f8577c71c0 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8576bdc30 con 0x55f8577bbfe0 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f6938414740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f85771d340 con 0x55f8577bc900 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f692e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f692f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 0x55f8577bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f692e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33192/0 (socket says 172.21.15.12:33192) 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f692e000640 1 -- 172.21.15.12:0/510103627 learned_addr learned my addr 172.21.15.12:0/510103627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.419+0000 7f692ea00640 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 -- 172.21.15.12:0/510103627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 msgr2=0x55f8577c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 -- 172.21.15.12:0/510103627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 msgr2=0x55f8577c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 -- 172.21.15.12:0/510103627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8576df940 con 0x55f8577bbfe0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f692f400640 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 0x55f8577bc3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6928009b90 tx=0x7f69280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8143e00f1948c179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f6927400640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 0x7f69140a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.420+0000 7f691f400640 1 -- 172.21.15.12:0/510103627 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692801c480 con 0x55f8577bbfe0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f691f400640 1 -- 172.21.15.12:0/510103627 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692801cad0 con 0x55f8577bbfe0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f691f400640 1 -- 172.21.15.12:0/510103627 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928011a60 con 0x55f8577bbfe0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6927400640 1 -- 172.21.15.12:0/1636152369 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f69180d1120 con 0x7f6930104210 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f692ea00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 0x7f69140a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 -- 172.21.15.12:0/510103627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 msgr2=0x55f8577bc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 0x55f8577bc3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6928009b90 tx=0x7f69280107a0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 -- 172.21.15.12:0/510103627 shutdown_connections 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 --2- 172.21.15.12:0/510103627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bbfe0 0x55f8577bc3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 -- 172.21.15.12:0/510103627 >> 172.21.15.12:0/510103627 conn(0x55f8577b84f0 msgr2=0x55f8577b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 -- 172.21.15.12:0/510103627 shutdown_connections 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f692ea00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 0x7f69140a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69200099a0 tx=0x7f6920004a80 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.421+0000 7f6938414740 1 -- 172.21.15.12:0/510103627 wait complete. 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.422+0000 7f6938414740 1 -- start start 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.422+0000 7f6938414740 1 Processor -- start 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577c71c0 0x55f8577c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f8577c9df0 con 0x55f8577bbfe0 2024-04-26T22:03:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8576a8c60 con 0x55f8577bc900 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f8577c9f70 con 0x55f8577c71c0 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f6938414740 10 client.0 _ll_register_callbacks cb 0x55f8576d4e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:54672/0 (socket says 172.21.15.12:54672) 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33208/0 (socket says 172.21.15.12:33208) 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692e000640 1 -- 172.21.15.12:0/3633491429 learned_addr learned my addr 172.21.15.12:0/3633491429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692f400640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577c71c0 0x55f8577c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692e000640 1 -- 172.21.15.12:0/3633491429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577c71c0 msgr2=0x55f8577c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.423+0000 7f692e000640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577c71c0 0x55f8577c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f692e000640 1 -- 172.21.15.12:0/3633491429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 msgr2=0x55f8577c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f692e000640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f692e000640 1 -- 172.21.15.12:0/3633491429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8576b7510 con 0x55f8577bc900 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f692e000640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6918099430 tx=0x7f691800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=773d6ae2e574d12c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6913e00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691809c3c0 con 0x55f8577bc900 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692801cad0 con 0x55f8577bc900 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f69180d3310 con 0x55f8577bc900 2024-04-26T22:03:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69180bd0b0 con 0x55f8577bc900 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6938414740 10 client.25457 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.424+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f857433210 con 0x55f8577bc900 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692801cad0 con 0x55f8577bc900 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f69180cebb0 con 0x55f8577bc900 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 10 client.25457 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 20 client.25457 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f8577d6ee0 0x55f8577d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 4 client.25457 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85772ef90 con 0x55f8577d6ee0 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.425+0000 7f692ea00640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f8577d6ee0 0x55f8577d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.426+0000 7f692ea00640 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f8577d6ee0 0x55f8577d92d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8819014122e0f6ef server_cookie=2e804c42981cc5b8 in_seq=0 out_seq=0 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.426+0000 7f6913e00640 10 client.25457 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6913e00640 1 -- 172.21.15.12:0/3633491429 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f85772ef90 con 0x55f8577d6ee0 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6913e00640 10 client.25457 handle_command_reply: tid=0 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 client.25457 shutdown 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 msgr2=0x55f8577c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6918099430 tx=0x7f691800cf20 comp rx=0 tx=0).stop 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 shutdown_connections 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f8577d6ee0 0x55f8577d92d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577c71c0 0x55f8577c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6913e00640 0 client.25457 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577c4660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 --2- 172.21.15.12:0/3633491429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 >> 172.21.15.12:0/3633491429 conn(0x55f8577b84f0 msgr2=0x55f8577be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 shutdown_connections 2024-04-26T22:03:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.427+0000 7f6938414740 1 -- 172.21.15.12:0/3633491429 wait complete. 2024-04-26T22:03:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.428+0000 7f6938414740 20 client.25457 trim_cache size 0 max 16384 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 Processor -- start 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 -- start start 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bbfe0 0x55f8577d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69301afd60 con 0x55f8577c71c0 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b25e0 con 0x55f8577bc900 2024-04-26T22:03:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.444+0000 7f6938414740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b03d0 con 0x55f8577bbfe0 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bbfe0 0x55f8577d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:54674/0 (socket says 172.21.15.12:54674) 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 -- 172.21.15.12:0/3892854297 learned_addr learned my addr 172.21.15.12:0/3892854297 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 -- 172.21.15.12:0/3892854297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bbfe0 msgr2=0x55f8577d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bbfe0 0x55f8577d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 -- 172.21.15.12:0/3892854297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 msgr2=0x55f8577deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.445+0000 7f692e000640 1 -- 172.21.15.12:0/3892854297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8574ff620 con 0x55f8577bc900 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.446+0000 7f692e000640 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69180bd250 tx=0x7f691800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7fc4592a6b08ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.446+0000 7f691ea00640 1 -- 172.21.15.12:0/3892854297 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691809c830 con 0x55f8577bc900 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.446+0000 7f691ea00640 1 -- 172.21.15.12:0/3892854297 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691809d160 con 0x55f8577bc900 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.446+0000 7f691ea00640 1 -- 172.21.15.12:0/3892854297 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918087370 con 0x55f8577bc900 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- 172.21.15.12:0/3892854297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 msgr2=0x55f8577be9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69180bd250 tx=0x7f691800ef70 comp rx=0 tx=0).stop 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- 172.21.15.12:0/3892854297 shutdown_connections 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577c71c0 0x55f8577deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577bc900 0x55f8577be9a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 --2- 172.21.15.12:0/3892854297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bbfe0 0x55f8577d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- 172.21.15.12:0/3892854297 >> 172.21.15.12:0/3892854297 conn(0x55f8577b84f0 msgr2=0x55f8577b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- 172.21.15.12:0/3892854297 shutdown_connections 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- 172.21.15.12:0/3892854297 wait complete. 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.447+0000 7f6938414740 1 -- start start 2024-04-26T22:03:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 Processor -- start 2024-04-26T22:03:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577c71c0 0x55f8577cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b0aa0 con 0x55f8577bbfe0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b1f10 con 0x55f8577c71c0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b1170 con 0x55f8577bc900 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f6938414740 10 client.0 _ll_register_callbacks cb 0x7f69180d48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.448+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33234/0 (socket says 172.21.15.12:33234) 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 -- 172.21.15.12:0/688447364 learned_addr learned my addr 172.21.15.12:0/688447364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692f400640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577c71c0 0x55f8577cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692e000640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 -- 172.21.15.12:0/688447364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 msgr2=0x55f8577cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 -- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577c71c0 msgr2=0x55f8577cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577c71c0 0x55f8577cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 -- 172.21.15.12:0/688447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8576dbcb0 con 0x55f8577bbfe0 2024-04-26T22:03:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692e000640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f692ea00640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6920003ae0 tx=0x7f692002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bdad63848fc83a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.449+0000 7f6913e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.450+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920036040 con 0x55f8577bbfe0 2024-04-26T22:03:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.450+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f6920030b70 con 0x55f8577bbfe0 2024-04-26T22:03:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.450+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691809d160 con 0x55f8577bbfe0 2024-04-26T22:03:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6920066050 con 0x55f8577bbfe0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 10 client.15336 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f85783d7d0 con 0x55f8577bbfe0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691809d160 con 0x55f8577bbfe0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6920004200 con 0x55f8577bbfe0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 10 client.15336 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 20 client.15336 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f85783dea0 0x55f8578402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 4 client.15336 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.451+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f857728e00 con 0x55f85783dea0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.452+0000 7f692e000640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f85783dea0 0x55f8578402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.452+0000 7f692e000640 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f85783dea0 0x55f8578402c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8baf025af095cf server_cookie=556172a00bd4606d in_seq=0 out_seq=0 2024-04-26T22:03:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.452+0000 7f6913e00640 10 client.15336 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.460+0000 7f6913e00640 1 -- 172.21.15.12:0/688447364 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f857728e00 con 0x55f85783dea0 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.460+0000 7f6913e00640 10 client.15336 handle_command_reply: tid=0 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.460+0000 7f6938414740 1 client.15336 shutdown 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.460+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 msgr2=0x55f8577cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.460+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6920003ae0 tx=0x7f692002a750 comp rx=0 tx=0).stop 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 shutdown_connections 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55f85783dea0 0x55f8578402c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8577c71c0 0x55f8577cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8577bc900 0x55f8577cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 --2- 172.21.15.12:0/688447364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8577bbfe0 0x55f8577cae10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6913e00640 0 client.15336 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 >> 172.21.15.12:0/688447364 conn(0x55f8577b84f0 msgr2=0x55f8577c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 shutdown_connections 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 1 -- 172.21.15.12:0/688447364 wait complete. 2024-04-26T22:03:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.461+0000 7f6938414740 20 client.15336 trim_cache size 0 max 16384 2024-04-26T22:03:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:03:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:03:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:03:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 msgr2=0x7f69140a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 0x7f69140a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69200099a0 tx=0x7f6920004a80 comp rx=0 tx=0).stop 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 msgr2=0x7f693019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 0x7f693019b720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f691800e7a0 tx=0x7f691800ec60 comp rx=0 tx=0).stop 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 shutdown_connections 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f69140a3e30 0x7f69140a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f693010cd10 0x7f693019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6930104210 0x7f693019b720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 --2- 172.21.15.12:0/1636152369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69301038f0 0x7f693019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 >> 172.21.15.12:0/1636152369 conn(0x7f69300fd150 msgr2=0x7f69301025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 shutdown_connections 2024-04-26T22:03:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:18.462+0000 7f692fe00640 1 -- 172.21.15.12:0/1636152369 wait complete. 2024-04-26T22:03:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:19 smithi077 ceph-mon[32280]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T22:03:19.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:19 smithi012 ceph-mon[28184]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T22:03:19.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:19 smithi154 ceph-mon[32578]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T22:03:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:21 smithi077 ceph-mon[32280]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:21.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:21 smithi012 ceph-mon[28184]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:21.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:21 smithi154 ceph-mon[32578]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:23.735 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:23 smithi077 ceph-mon[32280]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:23 smithi012 ceph-mon[28184]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:23.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:23 smithi154 ceph-mon[32578]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:03:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:25 smithi077 ceph-mon[32280]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:25 smithi012 ceph-mon[28184]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:25 smithi154 ceph-mon[32578]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:27 smithi077 ceph-mon[32280]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-04-26T22:03:27.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:27 smithi012 ceph-mon[28184]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-04-26T22:03:27.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:27 smithi154 ceph-mon[32578]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-04-26T22:03:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:29 smithi077 ceph-mon[32280]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:29.760 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:29 smithi012 ceph-mon[28184]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:29.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:29 smithi154 ceph-mon[32578]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 -- 172.21.15.12:0/3887646772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758104290 msgr2=0x7f775810c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 --2- 172.21.15.12:0/3887646772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758104290 0x7f775810c770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7744009900 tx=0x7f774402a490 comp rx=0 tx=0).stop 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 -- 172.21.15.12:0/3887646772 shutdown_connections 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 --2- 172.21.15.12:0/3887646772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f775810cd40 0x7f775810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 --2- 172.21.15.12:0/3887646772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758104290 0x7f775810c770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 --2- 172.21.15.12:0/3887646772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7758103970 0x7f7758103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 -- 172.21.15.12:0/3887646772 >> 172.21.15.12:0/3887646772 conn(0x7f77580fd150 msgr2=0x7f77580ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 -- 172.21.15.12:0/3887646772 shutdown_connections 2024-04-26T22:03:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.602+0000 7f775d800640 1 -- 172.21.15.12:0/3887646772 wait complete. 2024-04-26T22:03:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.604+0000 7f775d800640 1 Processor -- start 2024-04-26T22:03:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.604+0000 7f775d800640 1 -- start start 2024-04-26T22:03:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7758104290 0x7f77581994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f775810cd40 0x7f775819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7758112a10 con 0x7f7758103970 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7758112890 con 0x7f775810cd40 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775d800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7758112b90 con 0x7f7758104290 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f7757e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f7757e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35424/0 (socket says 172.21.15.12:35424) 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f7757e00640 1 -- 172.21.15.12:0/962724716 learned_addr learned my addr 172.21.15.12:0/962724716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f7757400640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7758104290 0x7f77581994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.605+0000 7f775ce00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f775810cd40 0x7f775819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 -- 172.21.15.12:0/962724716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7758104290 msgr2=0x7f77581994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7758104290 0x7f77581994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 -- 172.21.15.12:0/962724716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f775810cd40 msgr2=0x7f775819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f775810cd40 0x7f775819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 -- 172.21.15.12:0/962724716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775819df90 con 0x7f7758103970 2024-04-26T22:03:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7757e00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f774800ea60 tx=0x7f774800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7754c00640 1 -- 172.21.15.12:0/962724716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800cde0 con 0x7f7758103970 2024-04-26T22:03:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7754c00640 1 -- 172.21.15.12:0/962724716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77480035f0 con 0x7f7758103970 2024-04-26T22:03:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f7754c00640 1 -- 172.21.15.12:0/962724716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748010690 con 0x7f7758103970 2024-04-26T22:03:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.606+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775819e280 con 0x7f7758103970 2024-04-26T22:03:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.607+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77581a5b10 con 0x7f7758103970 2024-04-26T22:03:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.610+0000 7f7754c00640 1 -- 172.21.15.12:0/962724716 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f77480037a0 con 0x7f7758103970 2024-04-26T22:03:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 Processor -- start 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 -- start start 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6432300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f6441090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 0x55a3f64439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6397380 con 0x55a3f6432840 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6333c30 con 0x55a3f64415d0 2024-04-26T22:03:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.611+0000 7f7761bad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6393340 con 0x55a3f6431f20 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f6441090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7754c00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 0x7f773c0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f6441090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35426/0 (socket says 172.21.15.12:35426) 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f775ce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6432300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757e00640 1 -- 172.21.15.12:0/5652873 learned_addr learned my addr 172.21.15.12:0/5652873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 0x55a3f64439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7754c00640 1 -- 172.21.15.12:0/962724716 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f77480d1110 con 0x7f7758103970 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 -- 172.21.15.12:0/5652873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 msgr2=0x55a3f6432300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6432300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f775ce00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 0x7f773c0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 -- 172.21.15.12:0/5652873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 msgr2=0x55a3f6441090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f6441090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.612+0000 7f7757400640 1 -- 172.21.15.12:0/5652873 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6355940 con 0x55a3f64415d0 2024-04-26T22:03:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.613+0000 7f7757400640 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 0x55a3f64439c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f774402aa80 tx=0x7f7744004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb2f8d8df16c1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.613+0000 7f774d600640 1 -- 172.21.15.12:0/5652873 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744035040 con 0x55a3f64415d0 2024-04-26T22:03:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.613+0000 7f774d600640 1 -- 172.21.15.12:0/5652873 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77440054f0 con 0x55a3f64415d0 2024-04-26T22:03:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.613+0000 7f774d600640 1 -- 172.21.15.12:0/5652873 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440035f0 con 0x55a3f64415d0 2024-04-26T22:03:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- 172.21.15.12:0/5652873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 msgr2=0x55a3f64439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f775ce00640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 0x7f773c0a6370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7750005fd0 tx=0x7f7750004ab0 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 0x55a3f64439c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f774402aa80 tx=0x7f7744004190 comp rx=0 tx=0).stop 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- 172.21.15.12:0/5652873 shutdown_connections 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f64415d0 0x55a3f64439c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f6441090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 --2- 172.21.15.12:0/5652873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6432300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- 172.21.15.12:0/5652873 >> 172.21.15.12:0/5652873 conn(0x55a3f642e4f0 msgr2=0x55a3f642e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- 172.21.15.12:0/5652873 shutdown_connections 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- 172.21.15.12:0/5652873 wait complete. 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.614+0000 7f7761bad740 1 -- start start 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 Processor -- start 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f643e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f64415d0 0x55a3f643ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6444140 con 0x55a3f6432840 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f631ec60 con 0x55a3f6431f20 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a3f64442c0 con 0x55a3f64415d0 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7761bad740 10 client.0 _ll_register_callbacks cb 0x55a3f634ae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7757400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7757400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35430/0 (socket says 172.21.15.12:35430) 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7757400640 1 -- 172.21.15.12:0/266383975 learned_addr learned my addr 172.21.15.12:0/266383975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.615+0000 7f7757e00640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f643e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f775ce00640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f64415d0 0x55a3f643ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 -- 172.21.15.12:0/266383975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f64415d0 msgr2=0x55a3f643ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f64415d0 0x55a3f643ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 -- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 msgr2=0x55a3f643e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f643e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 -- 172.21.15.12:0/266383975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f632d510 con 0x55a3f6432840 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757e00640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f643e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7757400640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7744005ab0 tx=0x7f7744000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b72e357998dd860 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7742000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.616+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744035040 con 0x55a3f6432840 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77440054f0 con 0x55a3f6432840 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f7744004e50 con 0x55a3f6432840 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7744009520 con 0x55a3f6432840 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7761bad740 10 client.15348 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a3f60a9210 con 0x55a3f6432840 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77440054f0 con 0x55a3f6432840 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.617+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7744055030 con 0x55a3f6432840 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 10 client.15348 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 10 client.15348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 20 client.15348 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f6451140 0x55a3f6453530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 4 client.15348 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f63a4f90 con 0x55a3f6451140 2024-04-26T22:03:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.618+0000 7f7757e00640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f6451140 0x55a3f6453530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.619+0000 7f7742000640 10 client.15348 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.619+0000 7f7757e00640 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f6451140 0x55a3f6453530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6f93945ac668be server_cookie=e4fcdb1f153bf0fb in_seq=0 out_seq=0 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.619+0000 7f7742000640 1 -- 172.21.15.12:0/266383975 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3f63a4f90 con 0x55a3f6451140 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.619+0000 7f7742000640 10 client.15348 handle_command_reply: tid=0 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 client.15348 shutdown 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 msgr2=0x55a3f643ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7744005ab0 tx=0x7f7744000bc0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 shutdown_connections 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f6451140 0x55a3f6453530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f64415d0 0x55a3f643ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f6432840 0x55a3f643ea40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 --2- 172.21.15.12:0/266383975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f643e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7742000640 0 client.15348 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 >> 172.21.15.12:0/266383975 conn(0x55a3f642e4f0 msgr2=0x55a3f6434270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 shutdown_connections 2024-04-26T22:03:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 1 -- 172.21.15.12:0/266383975 wait complete. 2024-04-26T22:03:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.620+0000 7f7761bad740 20 client.15348 trim_cache size 0 max 16384 2024-04-26T22:03:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.637+0000 7f7761bad740 1 Processor -- start 2024-04-26T22:03:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.637+0000 7f7761bad740 1 -- start start 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6440c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6432840 0x55a3f6456550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6456a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77581ab8e0 con 0x55a3f64415d0 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77581ae160 con 0x55a3f6432840 2024-04-26T22:03:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7761bad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77581abf50 con 0x55a3f6431f20 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f775ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6456a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7757e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6440c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f775ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6456a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35434/0 (socket says 172.21.15.12:35434) 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f775ce00640 1 -- 172.21.15.12:0/1077743347 learned_addr learned my addr 172.21.15.12:0/1077743347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7757400640 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6432840 0x55a3f6456550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7757e00640 1 -- 172.21.15.12:0/1077743347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6432840 msgr2=0x55a3f6456550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.638+0000 7f7757e00640 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6432840 0x55a3f6456550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f7757e00640 1 -- 172.21.15.12:0/1077743347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 msgr2=0x55a3f6456a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f7757e00640 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6456a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f7757e00640 1 -- 172.21.15.12:0/1077743347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6175620 con 0x55a3f6431f20 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f7757e00640 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6440c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f77480ce390 tx=0x7f77480d3560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fcd4e50c06c78b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f774cc00640 1 -- 172.21.15.12:0/1077743347 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774809ca50 con 0x55a3f6431f20 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f774cc00640 1 -- 172.21.15.12:0/1077743347 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77480a0e00 con 0x55a3f6431f20 2024-04-26T22:03:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.639+0000 7f774cc00640 1 -- 172.21.15.12:0/1077743347 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480d3320 con 0x55a3f6431f20 2024-04-26T22:03:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 -- 172.21.15.12:0/1077743347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 msgr2=0x55a3f6440c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6440c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f77480ce390 tx=0x7f77480d3560 comp rx=0 tx=0).stop 2024-04-26T22:03:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 -- 172.21.15.12:0/1077743347 shutdown_connections 2024-04-26T22:03:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6456a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6432840 0x55a3f6456550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 --2- 172.21.15.12:0/1077743347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6431f20 0x55a3f6440c30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 -- 172.21.15.12:0/1077743347 >> 172.21.15.12:0/1077743347 conn(0x55a3f642e4f0 msgr2=0x55a3f644ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 -- 172.21.15.12:0/1077743347 shutdown_connections 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.640+0000 7f7761bad740 1 -- 172.21.15.12:0/1077743347 wait complete. 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 -- start start 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 Processor -- start 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f6453ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6432840 0x55a3f6454410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77581ac620 con 0x55a3f64415d0 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77581ada90 con 0x55a3f6431f20 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77581accf0 con 0x55a3f6432840 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.641+0000 7f7761bad740 10 client.0 _ll_register_callbacks cb 0x7f7748003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f7757e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f6453ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35446/0 (socket says 172.21.15.12:35446) 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 -- 172.21.15.12:0/2572434531 learned_addr learned my addr 172.21.15.12:0/2572434531 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f7757400640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6432840 0x55a3f6454410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 -- 172.21.15.12:0/2572434531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6432840 msgr2=0x55a3f6454410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6432840 0x55a3f6454410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 -- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 msgr2=0x55a3f6453ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f6453ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 -- 172.21.15.12:0/2572434531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f6351cb0 con 0x55a3f64415d0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f775ce00640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7750012b60 tx=0x7f7750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5869ac40a1b26f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.642+0000 7f7742000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775001a070 con 0x55a3f64415d0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f7750014240 con 0x55a3f64415d0 2024-04-26T22:03:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77480a0e00 con 0x55a3f64415d0 2024-04-26T22:03:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7750011a20 con 0x55a3f64415d0 2024-04-26T22:03:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7761bad740 10 client.15354 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a3f64b7440 con 0x55a3f64415d0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.643+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77480a0e00 con 0x55a3f64415d0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f775001d070 con 0x55a3f64415d0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 10 client.15354 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 20 client.15354 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f64b7bf0 0x55a3f64ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 4 client.15354 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3f639ee00 con 0x55a3f64b7bf0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.644+0000 7f7757e00640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f64b7bf0 0x55a3f64ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.645+0000 7f7757e00640 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f64b7bf0 0x55a3f64ba010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f633c381aac6cc server_cookie=c1b003000e6d352a in_seq=0 out_seq=0 2024-04-26T22:03:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.645+0000 7f7742000640 10 client.15354 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7742000640 1 -- 172.21.15.12:0/2572434531 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3f639ee00 con 0x55a3f64b7bf0 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7742000640 10 client.15354 handle_command_reply: tid=0 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 client.15354 shutdown 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 msgr2=0x55a3f6454950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7750012b60 tx=0x7f7750004a40 comp rx=0 tx=0).stop 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 shutdown_connections 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55a3f64b7bf0 0x55a3f64ba010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3f64415d0 0x55a3f6454950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3f6432840 0x55a3f6454410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7742000640 0 client.15354 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 --2- 172.21.15.12:0/2572434531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6431f20 0x55a3f6453ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 >> 172.21.15.12:0/2572434531 conn(0x55a3f642e4f0 msgr2=0x55a3f644c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 shutdown_connections 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 1 -- 172.21.15.12:0/2572434531 wait complete. 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.649+0000 7f7761bad740 20 client.15354 trim_cache size 0 max 16384 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:03:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 msgr2=0x7f773c0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 0x7f773c0a6370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7750005fd0 tx=0x7f7750004ab0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 msgr2=0x7f7758198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f774800ea60 tx=0x7f774800ef20 comp rx=0 tx=0).stop 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 shutdown_connections 2024-04-26T22:03:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f773c0a3eb0 0x7f773c0a6370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f775810cd40 0x7f775819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7758104290 0x7f77581994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 --2- 172.21.15.12:0/962724716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7758103970 0x7f7758198fa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 >> 172.21.15.12:0/962724716 conn(0x7f77580fd150 msgr2=0x7f77580fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 shutdown_connections 2024-04-26T22:03:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:31.650+0000 7f775d800640 1 -- 172.21.15.12:0/962724716 wait complete. 2024-04-26T22:03:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:31 smithi077 ceph-mon[32280]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:31.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:31 smithi012 ceph-mon[28184]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:31 smithi154 ceph-mon[32578]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:33 smithi012 ceph-mon[28184]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:33.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:33 smithi154 ceph-mon[32578]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:33 smithi077 ceph-mon[32280]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T22:03:35.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:35 smithi012 ceph-mon[28184]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T22:03:35.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:35 smithi154 ceph-mon[32578]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T22:03:35.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:35 smithi077 ceph-mon[32280]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T22:03:37.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:37 smithi012 ceph-mon[28184]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:37 smithi154 ceph-mon[32578]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:37.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:37 smithi077 ceph-mon[32280]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:39.784 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:39 smithi077 ceph-mon[32280]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:39.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:39 smithi012 ceph-mon[28184]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:39.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:39 smithi154 ceph-mon[32578]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:41.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:41 smithi012 ceph-mon[28184]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:41.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:41 smithi154 ceph-mon[32578]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:41.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:41 smithi077 ceph-mon[32280]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:43.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:43 smithi012 ceph-mon[28184]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:43.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:43 smithi154 ceph-mon[32578]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:43 smithi077 ceph-mon[32280]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 -- 172.21.15.12:0/2382744804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 msgr2=0x7fd20410f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 --2- 172.21.15.12:0/2382744804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd20410f1b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009be0 tx=0x7fd1fc02a750 comp rx=0 tx=0).stop 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 -- 172.21.15.12:0/2382744804 shutdown_connections 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 --2- 172.21.15.12:0/2382744804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd20410f1b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 --2- 172.21.15.12:0/2382744804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd204104210 0x7fd20410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 --2- 172.21.15.12:0/2382744804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2041038f0 0x7fd204103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 -- 172.21.15.12:0/2382744804 >> 172.21.15.12:0/2382744804 conn(0x7fd2040fd150 msgr2=0x7fd2040ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 -- 172.21.15.12:0/2382744804 shutdown_connections 2024-04-26T22:03:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.840+0000 7fd203e00640 1 -- 172.21.15.12:0/2382744804 wait complete. 2024-04-26T22:03:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 Processor -- start 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 -- start start 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd204104210 0x7fd204101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041129c0 con 0x7fd2041038f0 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112840 con 0x7fd20410cd10 2024-04-26T22:03:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.842+0000 7fd203e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112b40 con 0x7fd204104210 2024-04-26T22:03:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd202a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd202a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60008/0 (socket says 172.21.15.12:60008) 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd202a00640 1 -- 172.21.15.12:0/3868325362 learned_addr learned my addr 172.21.15.12:0/3868325362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:34864/0 (socket says 172.21.15.12:34864) 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd202000640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd204104210 0x7fd204101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 -- 172.21.15.12:0/3868325362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd204104210 msgr2=0x7fd204101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd204104210 0x7fd204101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 -- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 msgr2=0x7fd204101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd203400640 1 -- 172.21.15.12:0/3868325362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041ab8c0 con 0x7fd20410cd10 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.843+0000 7fd202a00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.844+0000 7fd203400640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009b30 tx=0x7fd1fc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.844+0000 7fd208a00640 1 -- 172.21.15.12:0/3868325362 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc041020 con 0x7fd20410cd10 2024-04-26T22:03:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.844+0000 7fd208a00640 1 -- 172.21.15.12:0/3868325362 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc0318f0 con 0x7fd20410cd10 2024-04-26T22:03:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.844+0000 7fd208a00640 1 -- 172.21.15.12:0/3868325362 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc036040 con 0x7fd20410cd10 2024-04-26T22:03:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.844+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041aba90 con 0x7fd20410cd10 2024-04-26T22:03:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.845+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041abf30 con 0x7fd20410cd10 2024-04-26T22:03:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.849+0000 7fd20cc5d740 1 Processor -- start 2024-04-26T22:03:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.850+0000 7fd208a00640 1 -- 172.21.15.12:0/3868325362 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7fd1fc044020 con 0x7fd20410cd10 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.850+0000 7fd20cc5d740 1 -- start start 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.850+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc86d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.850+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.850+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc87a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610bc7d2380 con 0x5610bc86cfe0 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610bc76ec30 con 0x5610bc86d900 2024-04-26T22:03:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5610bc7ce340 con 0x5610bc8781c0 2024-04-26T22:03:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:34880/0 (socket says 172.21.15.12:34880) 2024-04-26T22:03:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 -- 172.21.15.12:0/3424403769 learned_addr learned my addr 172.21.15.12:0/3424403769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202000640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc86d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202000640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc87a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 -- 172.21.15.12:0/3424403769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 msgr2=0x5610bc87a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc87a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 -- 172.21.15.12:0/3424403769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 msgr2=0x5610bc86d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc86d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202a00640 1 -- 172.21.15.12:0/3424403769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610bc790940 con 0x5610bc86d900 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.851+0000 7fd202000640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc87a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd202a00640 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4011630 tx=0x7fd1f4011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103c1f29f553d272 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd208a00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 0x7fd1e80a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd1efe00640 1 -- 172.21.15.12:0/3424403769 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f401e070 con 0x5610bc86d900 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd208a00640 1 -- 172.21.15.12:0/3868325362 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd1fc0f1540 con 0x7fd20410cd10 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd1efe00640 1 -- 172.21.15.12:0/3424403769 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f400fba0 con 0x5610bc86d900 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.852+0000 7fd1efe00640 1 -- 172.21.15.12:0/3424403769 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4019c80 con 0x5610bc86d900 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd202000640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 0x7fd1e80a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3424403769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 msgr2=0x5610bc877c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4011630 tx=0x7fd1f4011af0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3424403769 shutdown_connections 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc87a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc877c80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3424403769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc86d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.853+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3424403769 >> 172.21.15.12:0/3424403769 conn(0x5610bc8694f0 msgr2=0x5610bc8698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.854+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3424403769 shutdown_connections 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.854+0000 7fd202000640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 0x7fd1e80a6370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x5610bc86a680 tx=0x7fd1f0005a20 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.854+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3424403769 wait complete. 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.854+0000 7fd20cc5d740 1 -- start start 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 Processor -- start 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc875070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc8755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610bc87ae40 con 0x5610bc86cfe0 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610bc759c60 con 0x5610bc86d900 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5610bc87afc0 con 0x5610bc8781c0 2024-04-26T22:03:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.855+0000 7fd20cc5d740 10 client.0 _ll_register_callbacks cb 0x5610bc785e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc875070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd203400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd203400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50756/0 (socket says 172.21.15.12:50756) 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd203400640 1 -- 172.21.15.12:0/4057611501 learned_addr learned my addr 172.21.15.12:0/4057611501 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202000640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc8755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 -- 172.21.15.12:0/4057611501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 msgr2=0x5610bc875af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 -- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 msgr2=0x5610bc8755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc8755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd202a00640 1 -- 172.21.15.12:0/4057611501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610bc768510 con 0x5610bc86cfe0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.856+0000 7fd203400640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd202a00640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc875070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd1f401c040 tx=0x7fd1f4009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f8f9636f98b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd1ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4026020 con 0x5610bc86cfe0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd1f400fd50 con 0x5610bc86cfe0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f400fba0 con 0x5610bc86cfe0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd1f4031050 con 0x5610bc86cfe0 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd20cc5d740 10 client.15366 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.857+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5610bc4e4210 con 0x5610bc86cfe0 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f400fba0 con 0x5610bc86cfe0 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd1f4021050 con 0x5610bc86cfe0 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 10 client.15366 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 20 client.15366 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc887e10 0x5610bc88a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 4 client.15366 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.858+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610bc7dff90 con 0x5610bc887e10 2024-04-26T22:03:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.859+0000 7fd202000640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc887e10 0x5610bc88a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.859+0000 7fd202000640 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc887e10 0x5610bc88a200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3e459f3da222cc server_cookie=cb26dcac9c2b703c in_seq=0 out_seq=0 2024-04-26T22:03:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.860+0000 7fd1ecc00640 10 client.15366 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.860+0000 7fd1ecc00640 1 -- 172.21.15.12:0/4057611501 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610bc7dff90 con 0x5610bc887e10 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.860+0000 7fd1ecc00640 10 client.15366 handle_command_reply: tid=0 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.860+0000 7fd20cc5d740 1 client.15366 shutdown 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 msgr2=0x5610bc875070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc875070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd1f401c040 tx=0x7fd1f4009f10 comp rx=0 tx=0).stop 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 shutdown_connections 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc887e10 0x5610bc88a200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc8781c0 0x5610bc875af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86d900 0x5610bc8755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/4057611501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc875070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 >> 172.21.15.12:0/4057611501 conn(0x5610bc8694f0 msgr2=0x5610bc86f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd1ecc00640 0 client.15366 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 shutdown_connections 2024-04-26T22:03:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 1 -- 172.21.15.12:0/4057611501 wait complete. 2024-04-26T22:03:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.861+0000 7fd20cc5d740 20 client.15366 trim_cache size 0 max 16384 2024-04-26T22:03:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.879+0000 7fd20cc5d740 1 Processor -- start 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.879+0000 7fd20cc5d740 1 -- start start 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc8777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc8781c0 0x5610bc874050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b4120 con 0x5610bc86cfe0 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b69a0 con 0x5610bc8781c0 2024-04-26T22:03:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b4790 con 0x5610bc86d900 2024-04-26T22:03:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc8777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:50760/0 (socket says 172.21.15.12:50760) 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202000640 1 -- 172.21.15.12:0/3243848805 learned_addr learned my addr 172.21.15.12:0/3243848805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc8777a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60044/0 (socket says 172.21.15.12:60044) 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd203400640 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc8781c0 0x5610bc874050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202000640 1 -- 172.21.15.12:0/3243848805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc8781c0 msgr2=0x5610bc874050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.880+0000 7fd202000640 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc8781c0 0x5610bc874050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd202000640 1 -- 172.21.15.12:0/3243848805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 msgr2=0x5610bc8777a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd202000640 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc8777a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd202000640 1 -- 172.21.15.12:0/3243848805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610bc5b0620 con 0x5610bc86d900 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd202000640 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd20419d050 tx=0x7fd1f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb56f462263b078 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd1ef400640 1 -- 172.21.15.12:0/3243848805 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00041a0 con 0x5610bc86d900 2024-04-26T22:03:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd1ef400640 1 -- 172.21.15.12:0/3243848805 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f002c260 con 0x5610bc86d900 2024-04-26T22:03:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.881+0000 7fd1ef400640 1 -- 172.21.15.12:0/3243848805 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0011580 con 0x5610bc86d900 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3243848805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 msgr2=0x5610bc883d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd20419d050 tx=0x7fd1f00046c0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3243848805 shutdown_connections 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc8781c0 0x5610bc874050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc883d00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/3243848805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc86cfe0 0x5610bc8777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.882+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3243848805 >> 172.21.15.12:0/3243848805 conn(0x5610bc8694f0 msgr2=0x5610bc873940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.883+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3243848805 shutdown_connections 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.883+0000 7fd20cc5d740 1 -- 172.21.15.12:0/3243848805 wait complete. 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.883+0000 7fd20cc5d740 1 -- start start 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.883+0000 7fd20cc5d740 1 Processor -- start 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc87ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 0x5610bc8e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b4e60 con 0x5610bc8781c0 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b62d0 con 0x5610bc86cfe0 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b5530 con 0x5610bc86d900 2024-04-26T22:03:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd20cc5d740 10 client.0 _ll_register_callbacks cb 0x7fd1f401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.884+0000 7fd202a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:34914/0 (socket says 172.21.15.12:34914) 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 -- 172.21.15.12:0/766827645 learned_addr learned my addr 172.21.15.12:0/766827645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd203400640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 0x5610bc8e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202000640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc87ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 -- 172.21.15.12:0/766827645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 msgr2=0x5610bc87ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc87ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 -- 172.21.15.12:0/766827645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 msgr2=0x5610bc8e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 0x5610bc8e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 -- 172.21.15.12:0/766827645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610bc78ccb0 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd203400640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 0x5610bc8e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd202a00640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006010 tx=0x7fd1f4004ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5068ae214f5ed343 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd1ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.885+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4026040 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd1f400fc10 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f002c260 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd1f401e040 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd20cc5d740 10 client.25499 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5610bc8ed930 con 0x5610bc86cfe0 2024-04-26T22:03:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f002c260 con 0x5610bc86cfe0 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.886+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd1f4021070 con 0x5610bc86cfe0 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 10 client.25499 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 20 client.25499 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc8ee040 0x5610bc8f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 4 client.25499 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:03:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.887+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610bc86fe70 con 0x5610bc8ee040 2024-04-26T22:03:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.888+0000 7fd202000640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc8ee040 0x5610bc8f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.888+0000 7fd202000640 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc8ee040 0x5610bc8f0460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09da3d5386b2384 server_cookie=1b402021b0fddfa in_seq=0 out_seq=0 2024-04-26T22:03:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.888+0000 7fd1ecc00640 10 client.25499 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.892+0000 7fd1ecc00640 1 -- 172.21.15.12:0/766827645 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610bc86fe70 con 0x5610bc8ee040 2024-04-26T22:03:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.892+0000 7fd1ecc00640 10 client.25499 handle_command_reply: tid=0 2024-04-26T22:03:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.893+0000 7fd20cc5d740 1 client.25499 shutdown 2024-04-26T22:03:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 msgr2=0x5610bc87b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006010 tx=0x7fd1f4004ca0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 shutdown_connections 2024-04-26T22:03:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5610bc8ee040 0x5610bc8f0460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610bc8781c0 0x5610bc8e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5610bc86d900 0x5610bc87ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 --2- 172.21.15.12:0/766827645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610bc86cfe0 0x5610bc87b4c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 >> 172.21.15.12:0/766827645 conn(0x5610bc8694f0 msgr2=0x5610bc8741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd1ecc00640 0 client.25499 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 shutdown_connections 2024-04-26T22:03:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.894+0000 7fd20cc5d740 1 -- 172.21.15.12:0/766827645 wait complete. 2024-04-26T22:03:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.895+0000 7fd20cc5d740 20 client.25499 trim_cache size 0 max 16384 2024-04-26T22:03:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:03:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:03:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:03:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:03:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.895+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 msgr2=0x7fd1e80a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.895+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 0x7fd1e80a6370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x5610bc86a680 tx=0x7fd1f0005a20 comp rx=0 tx=0).stop 2024-04-26T22:03:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.895+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 msgr2=0x7fd204101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.895+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009b30 tx=0x7fd1fc003ba0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 shutdown_connections 2024-04-26T22:03:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd1e80a3eb0 0x7fd1e80a6370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20410cd10 0x7fd204101fc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd204104210 0x7fd204101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 --2- 172.21.15.12:0/3868325362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2041038f0 0x7fd204101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.896+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 >> 172.21.15.12:0/3868325362 conn(0x7fd2040fd150 msgr2=0x7fd2040fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.897+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 shutdown_connections 2024-04-26T22:03:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:44.897+0000 7fd203e00640 1 -- 172.21.15.12:0/3868325362 wait complete. 2024-04-26T22:03:45.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:45 smithi012 ceph-mon[28184]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:45.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:45 smithi154 ceph-mon[32578]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:45 smithi077 ceph-mon[32280]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:46.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:46 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:46.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:46 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:46 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:47 smithi012 ceph-mon[28184]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T22:03:47.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:47 smithi154 ceph-mon[32578]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T22:03:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:47 smithi077 ceph-mon[32280]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T22:03:48.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:48 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:48.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:48 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:48.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:48 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:03:48.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:48 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:48.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:48 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:48.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:48 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:03:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:48 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:48 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:48.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:48 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:03:49.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:49 smithi012 ceph-mon[28184]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:49.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:49 smithi154 ceph-mon[32578]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:49.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:49 smithi077 ceph-mon[32280]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:51 smithi012 ceph-mon[28184]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:51.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:51 smithi154 ceph-mon[32578]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:51.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:51 smithi077 ceph-mon[32280]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:53.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:53 smithi012 ceph-mon[28184]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:53.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:53 smithi154 ceph-mon[32578]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:53.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:53 smithi077 ceph-mon[32280]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:55 smithi012 ceph-mon[28184]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:55 smithi154 ceph-mon[32578]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:55 smithi077 ceph-mon[32280]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:57 smithi012 ceph-mon[28184]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:57.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:57 smithi154 ceph-mon[32578]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:57 smithi077 ceph-mon[32280]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.087+0000 7f2e93e00640 1 -- 172.21.15.12:0/3334514689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 msgr2=0x7f2e94103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.087+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3334514689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e94103cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009860 tx=0x7f2e8402a3d0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 -- 172.21.15.12:0/3334514689 shutdown_connections 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3334514689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e9410cd10 0x7f2e9410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3334514689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e94104210 0x7f2e9410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3334514689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e94103cd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 -- 172.21.15.12:0/3334514689 >> 172.21.15.12:0/3334514689 conn(0x7f2e940fd150 msgr2=0x7f2e940ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 -- 172.21.15.12:0/3334514689 shutdown_connections 2024-04-26T22:03:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.088+0000 7f2e93e00640 1 -- 172.21.15.12:0/3334514689 wait complete. 2024-04-26T22:03:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.089+0000 7f2e93e00640 1 Processor -- start 2024-04-26T22:03:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 -- start start 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e9419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 0x7f2e9419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94112a10 con 0x7f2e9410cd10 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94112890 con 0x7f2e941038f0 2024-04-26T22:03:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.090+0000 7f2e93e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94112b90 con 0x7f2e94104210 2024-04-26T22:03:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e93400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 0x7f2e9419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:37004/0 (socket says 172.21.15.12:37004) 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e93400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 0x7f2e9419fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55572/0 (socket says 172.21.15.12:55572) 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 -- 172.21.15.12:0/3461391865 learned_addr learned my addr 172.21.15.12:0/3461391865 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e9419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 -- 172.21.15.12:0/3461391865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 msgr2=0x7f2e9419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e9419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 -- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 msgr2=0x7f2e9419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 0x7f2e9419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92000640 1 -- 172.21.15.12:0/3461391865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e941a0140 con 0x7f2e94104210 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.091+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e9419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e92000640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e910 tx=0x7f2e8000edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e8b400640 1 -- 172.21.15.12:0/3461391865 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000cd80 con 0x7f2e94104210 2024-04-26T22:03:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e941a0430 con 0x7f2e94104210 2024-04-26T22:03:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e8b400640 1 -- 172.21.15.12:0/3461391865 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e800035f0 con 0x7f2e94104210 2024-04-26T22:03:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e8b400640 1 -- 172.21.15.12:0/3461391865 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80010690 con 0x7f2e94104210 2024-04-26T22:03:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.092+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941a7cc0 con 0x7f2e94104210 2024-04-26T22:03:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.097+0000 7f2e9cbaf740 1 Processor -- start 2024-04-26T22:03:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.097+0000 7f2e9cbaf740 1 -- start start 2024-04-26T22:03:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.097+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df8427b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.097+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e8b400640 1 -- 172.21.15.12:0/3461391865 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 138879+0+0 (secure 0 0 0) 0x7f2e800037a0 con 0x7f2e94104210 2024-04-26T22:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df84290dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55df841e0380 con 0x55df8427af20 2024-04-26T22:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df8417cc30 con 0x55df8428e9d0 2024-04-26T22:03:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55df841dc340 con 0x55df8427b840 2024-04-26T22:03:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e92a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e92a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:37012/0 (socket says 172.21.15.12:37012) 2024-04-26T22:03:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e92a00640 1 -- 172.21.15.12:0/3127234923 learned_addr learned my addr 172.21.15.12:0/3127234923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e93400640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df8427b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e92a00640 1 -- 172.21.15.12:0/3127234923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 msgr2=0x55df84290dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.098+0000 7f2e92000640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df84290dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df84290dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92a00640 1 -- 172.21.15.12:0/3127234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 msgr2=0x55df8427b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df8427b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92a00640 1 -- 172.21.15.12:0/3127234923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8419e940 con 0x55df8427b840 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e93400640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df8427b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92000640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df84290dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e84034000 tx=0x7f2e84030860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7da16e4fec1e9c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.099+0000 7f2e7fe00640 1 -- 172.21.15.12:0/3127234923 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84030d00 con 0x55df8427b840 2024-04-26T22:03:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.100+0000 7f2e7fe00640 1 -- 172.21.15.12:0/3127234923 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84041030 con 0x55df8427b840 2024-04-26T22:03:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.100+0000 7f2e7fe00640 1 -- 172.21.15.12:0/3127234923 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004920 con 0x55df8427b840 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.100+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3127234923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 msgr2=0x55df8428e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.100+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e84034000 tx=0x7f2e84030860 comp rx=0 tx=0).stop 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3127234923 shutdown_connections 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df84290dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427b840 0x55df8428e490 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3127234923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df8427b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3127234923 >> 172.21.15.12:0/3127234923 conn(0x55df842774f0 msgr2=0x55df842778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3127234923 shutdown_connections 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3127234923 wait complete. 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e8b400640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 0x7f2e780a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.101+0000 7f2e8b400640 1 -- 172.21.15.12:0/3461391865 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f2e800d0fe0 con 0x7f2e94104210 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.102+0000 7f2e9cbaf740 1 -- start start 2024-04-26T22:03:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.102+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 0x7f2e780a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.102+0000 7f2e9cbaf740 1 Processor -- start 2024-04-26T22:03:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427af20 0x55df8428b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55df84291540 con 0x55df8427b840 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df84167c60 con 0x55df8428e9d0 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55df842916c0 con 0x55df8427af20 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e9cbaf740 10 client.0 _ll_register_callbacks cb 0x55df84193e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 0x7f2e780a6370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e84004600 tx=0x7f2e8403f040 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e93400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e93400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:47876/0 (socket says 172.21.15.12:47876) 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e93400640 1 -- 172.21.15.12:0/3233489884 learned_addr learned my addr 172.21.15.12:0/3233489884 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.103+0000 7f2e92000640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427af20 0x55df8428b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 -- 172.21.15.12:0/3233489884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427af20 msgr2=0x55df8428b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427af20 0x55df8428b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 -- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 msgr2=0x55df8428c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 -- 172.21.15.12:0/3233489884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df84176510 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e93400640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e92000640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428be20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2e800ce1e0 tx=0x7f2e8009c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20564ac381729c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.104+0000 7f2e7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.105+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800a0d10 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.105+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f2e800d3110 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.105+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e84041030 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.105+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e800a55d0 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.105+0000 7f2e9cbaf740 10 client.15384 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.106+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55df83ef2210 con 0x55df8427b840 2024-04-26T22:03:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.106+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84041030 con 0x55df8427b840 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.106+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2e8009bdb0 con 0x55df8427b840 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.106+0000 7f2e9cbaf740 10 client.15384 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.106+0000 7f2e9cbaf740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.107+0000 7f2e9cbaf740 20 client.15384 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.107+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df8429e600 0x55df842a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.107+0000 7f2e9cbaf740 4 client.15384 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.107+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df841edf90 con 0x55df8429e600 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.107+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df8429e600 0x55df842a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.108+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df8429e600 0x55df842a09f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf221f1255c6d7ce server_cookie=e220181bc9a0c488 in_seq=0 out_seq=0 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.108+0000 7f2e7cc00640 10 client.15384 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e7cc00640 1 -- 172.21.15.12:0/3233489884 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df841edf90 con 0x55df8429e600 2024-04-26T22:03:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e7cc00640 10 client.15384 handle_command_reply: tid=0 2024-04-26T22:03:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 client.15384 shutdown 2024-04-26T22:03:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 msgr2=0x55df8428be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428be20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2e800ce1e0 tx=0x7f2e8009c380 comp rx=0 tx=0).stop 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 shutdown_connections 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df8429e600 0x55df842a09f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e7cc00640 0 client.15384 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8428e9d0 0x55df8428c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428be20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.109+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3233489884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8427af20 0x55df8428b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.110+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 >> 172.21.15.12:0/3233489884 conn(0x55df842774f0 msgr2=0x55df8427d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.110+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 shutdown_connections 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.110+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3233489884 wait complete. 2024-04-26T22:03:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.110+0000 7f2e9cbaf740 20 client.15384 trim_cache size 0 max 16384 2024-04-26T22:03:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.141+0000 7f2e9cbaf740 1 Processor -- start 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.141+0000 7f2e9cbaf740 1 -- start start 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 0x55df8429a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df842a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941aba40 con 0x55df8427b840 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941ae260 con 0x55df8427af20 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941ac090 con 0x55df8428e9d0 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e92000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e92000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55584/0 (socket says 172.21.15.12:55584) 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e92000640 1 -- 172.21.15.12:0/3370170730 learned_addr learned my addr 172.21.15.12:0/3370170730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.142+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 0x55df8429a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e93400640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df842a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92a00640 1 -- 172.21.15.12:0/3370170730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 msgr2=0x55df842a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df842a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92a00640 1 -- 172.21.15.12:0/3370170730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 msgr2=0x55df8428d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92a00640 1 -- 172.21.15.12:0/3370170730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df842a0d60 con 0x55df8427af20 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.143+0000 7f2e92000640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.144+0000 7f2e93400640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df842a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.144+0000 7f2e92a00640 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 0x55df8429a440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402a410 tx=0x7f2e8404f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b895bce77af5dbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.144+0000 7f2e7f400640 1 -- 172.21.15.12:0/3370170730 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84065070 con 0x55df8427af20 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.144+0000 7f2e7f400640 1 -- 172.21.15.12:0/3370170730 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84041040 con 0x55df8427af20 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.144+0000 7f2e7f400640 1 -- 172.21.15.12:0/3370170730 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8403d700 con 0x55df8427af20 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3370170730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 msgr2=0x55df8429a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 0x55df8429a440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402a410 tx=0x7f2e8404f000 comp rx=0 tx=0).stop 2024-04-26T22:03:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3370170730 shutdown_connections 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df842a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427b840 0x55df8428d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/3370170730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427af20 0x55df8429a440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.145+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3370170730 >> 172.21.15.12:0/3370170730 conn(0x55df842774f0 msgr2=0x55df8428d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.146+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3370170730 shutdown_connections 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.146+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/3370170730 wait complete. 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.146+0000 7f2e9cbaf740 1 -- start start 2024-04-26T22:03:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 Processor -- start 2024-04-26T22:03:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df84289010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941ac720 con 0x55df8427af20 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941adb90 con 0x55df8427b840 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941acdf0 con 0x55df8428e9d0 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.147+0000 7f2e9cbaf740 10 client.0 _ll_register_callbacks cb 0x7f2e800984e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55590/0 (socket says 172.21.15.12:55590) 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:47896/0 (socket says 172.21.15.12:47896) 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92a00640 1 -- 172.21.15.12:0/2506791507 learned_addr learned my addr 172.21.15.12:0/2506791507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e93400640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df84289010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 -- 172.21.15.12:0/2506791507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 msgr2=0x55df84289010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df84289010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 -- 172.21.15.12:0/2506791507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 msgr2=0x55df84288590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92000640 1 -- 172.21.15.12:0/2506791507 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df83fbe620 con 0x55df8427b840 2024-04-26T22:03:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.148+0000 7f2e92a00640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e92000640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e8009c610 tx=0x7f2e8009ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1a43ae1a3a9deb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800a0d10 con 0x55df8427b840 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f2e80087290 con 0x55df8427b840 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e84041040 con 0x55df8427b840 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e8009c730 con 0x55df8427b840 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e9cbaf740 10 client.25514 fetch_fsmap learned FSMap version 21 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.149+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55df8419acb0 con 0x55df8427b840 2024-04-26T22:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84041040 con 0x55df8427b840 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2e800bbb70 con 0x55df8427b840 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 10 client.25514 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 20 client.25514 populate_metadata read hostname 'smithi012' 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df843050c0 0x55df843074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 4 client.25514 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.150+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df841e7e00 con 0x55df843050c0 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.151+0000 7f2e92a00640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df843050c0 0x55df843074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.151+0000 7f2e92a00640 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df843050c0 0x55df843074e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcbc6cf57a60586 server_cookie=2055b90c367945ef in_seq=0 out_seq=0 2024-04-26T22:03:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.151+0000 7f2e7cc00640 10 client.25514 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e7cc00640 1 -- 172.21.15.12:0/2506791507 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df841e7e00 con 0x55df843050c0 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e7cc00640 10 client.25514 handle_command_reply: tid=0 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 client.25514 shutdown 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 msgr2=0x55df84288ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e8009c610 tx=0x7f2e8009ae70 comp rx=0 tx=0).stop 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 shutdown_connections 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55df843050c0 0x55df843074e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55df8428e9d0 0x55df84289010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df8427b840 0x55df84288ad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e7cc00640 0 client.25514 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:03:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 --2- 172.21.15.12:0/2506791507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df8427af20 0x55df84288590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.157+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 >> 172.21.15.12:0/2506791507 conn(0x55df842774f0 msgr2=0x55df8427d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.158+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 shutdown_connections 2024-04-26T22:03:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.158+0000 7f2e9cbaf740 1 -- 172.21.15.12:0/2506791507 wait complete. 2024-04-26T22:03:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.158+0000 7f2e9cbaf740 20 client.25514 trim_cache size 0 max 16384 2024-04-26T22:03:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:03:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:03:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:03:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 msgr2=0x7f2e780a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 0x7f2e780a6370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e84004600 tx=0x7f2e8403f040 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 msgr2=0x7f2e9419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e910 tx=0x7f2e8000edd0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 shutdown_connections 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f2e780a3eb0 0x7f2e780a6370 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e9410cd10 0x7f2e9419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e94104210 0x7f2e9419b690 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000e910 tx=0x7f2e8000edd0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 --2- 172.21.15.12:0/3461391865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e941038f0 0x7f2e9419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 >> 172.21.15.12:0/3461391865 conn(0x7f2e940fd150 msgr2=0x7f2e94102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 shutdown_connections 2024-04-26T22:03:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:03:58.159+0000 7f2e93e00640 1 -- 172.21.15.12:0/3461391865 wait complete. 2024-04-26T22:03:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:03:59 smithi012 ceph-mon[28184]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:59.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:03:59 smithi154 ceph-mon[32578]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:03:59.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:03:59 smithi077 ceph-mon[32280]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T22:04:01.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:01 smithi012 ceph-mon[28184]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:01 smithi154 ceph-mon[32578]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:01 smithi077 ceph-mon[32280]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:03 smithi012 ceph-mon[28184]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:03 smithi154 ceph-mon[32578]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:03.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:03 smithi077 ceph-mon[32280]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:05 smithi012 ceph-mon[28184]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:04:05.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:05 smithi154 ceph-mon[32578]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:04:05.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:05 smithi077 ceph-mon[32280]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:04:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:07 smithi012 ceph-mon[28184]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:04:07.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:07 smithi154 ceph-mon[32578]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:04:07.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:07 smithi077 ceph-mon[32280]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:04:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:09 smithi012 ceph-mon[28184]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T22:04:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:09 smithi012 ceph-mon[28184]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T22:04:09.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:09 smithi154 ceph-mon[32578]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T22:04:09.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:09 smithi154 ceph-mon[32578]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T22:04:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:09 smithi077 ceph-mon[32280]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T22:04:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:09 smithi077 ceph-mon[32280]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T22:04:10.356 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 58% (7228/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 97% (11909/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T22:04:11.077 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T22:04:11.077 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.077 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T22:04:11.078 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.079 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T22:04:11.080 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-26T22:04:11.080 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T22:04:11.091 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T22:04:11.203 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T22:04:11.210 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T22:04:11.211 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T22:04:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 -- 172.21.15.12:0/1110804821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c103920 msgr2=0x7f633c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 --2- 172.21.15.12:0/1110804821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c103920 0x7f633c103d00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f632c009900 tx=0x7f632c02a410 comp rx=0 tx=0).stop 2024-04-26T22:04:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 -- 172.21.15.12:0/1110804821 shutdown_connections 2024-04-26T22:04:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 --2- 172.21.15.12:0/1110804821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c10cd90 0x7f633c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 --2- 172.21.15.12:0/1110804821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c104240 0x7f633c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 --2- 172.21.15.12:0/1110804821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c103920 0x7f633c103d00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 -- 172.21.15.12:0/1110804821 >> 172.21.15.12:0/1110804821 conn(0x7f633c0fd180 msgr2=0x7f633c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 -- 172.21.15.12:0/1110804821 shutdown_connections 2024-04-26T22:04:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.302+0000 7f633be00640 1 -- 172.21.15.12:0/1110804821 wait complete. 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 Processor -- start 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 -- start start 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 0x7f633c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 0x7f633c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f633c112a40 con 0x7f633c10cd90 2024-04-26T22:04:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.303+0000 7f633be00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1128c0 con 0x7f633c103920 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f633c112bc0 con 0x7f633c104240 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 0x7f633c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 0x7f633c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38446/0 (socket says 172.21.15.12:38446) 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 0x7f633c19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:46414/0 (socket says 172.21.15.12:46414) 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 0x7f633c19fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40270/0 (socket says 172.21.15.12:40270) 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 -- 172.21.15.12:0/2386816388 learned_addr learned my addr 172.21.15.12:0/2386816388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 -- 172.21.15.12:0/2386816388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 msgr2=0x7f633c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 0x7f633c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 -- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 msgr2=0x7f633c19fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 0x7f633c19fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.304+0000 7f633aa00640 1 -- 172.21.15.12:0/2386816388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c1a0150 con 0x7f633c103920 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f633aa00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f632c005ab0 tx=0x7f632c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f6333400640 1 -- 172.21.15.12:0/2386816388 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c03f070 con 0x7f633c103920 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c1a03e0 con 0x7f633c103920 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c1a7cd0 con 0x7f633c103920 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f6333400640 1 -- 172.21.15.12:0/2386816388 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632c031b60 con 0x7f633c103920 2024-04-26T22:04:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.305+0000 7f6333400640 1 -- 172.21.15.12:0/2386816388 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c049060 con 0x7f633c103920 2024-04-26T22:04:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.308+0000 7f634461f740 1 Processor -- start 2024-04-26T22:04:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.308+0000 7f634461f740 1 -- start start 2024-04-26T22:04:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.308+0000 7f6333400640 1 -- 172.21.15.12:0/2386816388 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f632c042070 con 0x7f633c103920 2024-04-26T22:04:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55823442a380 con 0x5582344c4fe0 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582343c6c30 con 0x5582344d01c0 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f634461f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558234426340 con 0x5582344c5900 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40272/0 (socket says 172.21.15.12:40272) 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 -- 172.21.15.12:0/3794580133 learned_addr learned my addr 172.21.15.12:0/3794580133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633a000640 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 -- 172.21.15.12:0/3794580133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 msgr2=0x5582344cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 -- 172.21.15.12:0/3794580133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 msgr2=0x5582344d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.309+0000 7f633b400640 1 -- 172.21.15.12:0/3794580133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582343e8940 con 0x5582344c4fe0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f633aa00640 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f633b400640 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6334009f50 tx=0x7f63340118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec62e6a870a2aad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f6327e00640 1 -- 172.21.15.12:0/3794580133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334004d20 con 0x5582344c4fe0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f6327e00640 1 -- 172.21.15.12:0/3794580133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334004050 con 0x5582344c4fe0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f6327e00640 1 -- 172.21.15.12:0/3794580133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334005260 con 0x5582344c4fe0 2024-04-26T22:04:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.310+0000 7f6333400640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 0x7f63200a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f6333400640 1 -- 172.21.15.12:0/2386816388 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f632c0f0e90 con 0x7f633c103920 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f633aa00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 0x7f63200a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 -- 172.21.15.12:0/3794580133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 msgr2=0x5582344c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6334009f50 tx=0x7f63340118d0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 -- 172.21.15.12:0/3794580133 shutdown_connections 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 --2- 172.21.15.12:0/3794580133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344c53c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 -- 172.21.15.12:0/3794580133 >> 172.21.15.12:0/3794580133 conn(0x5582344c14f0 msgr2=0x5582344c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f633aa00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 0x7f63200a6250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x5582344c6580 tx=0x7f632c0f4940 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 -- 172.21.15.12:0/3794580133 shutdown_connections 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.311+0000 7f634461f740 1 -- 172.21.15.12:0/3794580133 wait complete. 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 -- start start 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 Processor -- start 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5582344d2df0 con 0x5582344c4fe0 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582343b1c60 con 0x5582344d01c0 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.312+0000 7f634461f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5582344d2f70 con 0x5582344c5900 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f634461f740 10 client.0 _ll_register_callbacks cb 0x5582343dde50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40284/0 (socket says 172.21.15.12:40284) 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633b400640 1 -- 172.21.15.12:0/2078583239 learned_addr learned my addr 172.21.15.12:0/2078583239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633b400640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 -- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 msgr2=0x5582344cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 -- 172.21.15.12:0/2078583239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 msgr2=0x5582344cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633a000640 1 -- 172.21.15.12:0/2078583239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582343c0510 con 0x5582344c5900 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.313+0000 7f633b400640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f633a000640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cd5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5582344c2680 tx=0x7f6328012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0b1e5d61c4004a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f6324c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:04:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632800f8f0 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6334004050 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f632800fa90 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f632800fc90 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f634461f740 10 client.25519 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.314+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55823413c210 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334004050 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f632801c420 con 0x5582344c5900 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 10 client.25519 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 10 client.25519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 20 client.25519 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5582344dfee0 0x5582344e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 4 client.25519 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.315+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558234437f90 con 0x5582344dfee0 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.316+0000 7f633b400640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5582344dfee0 0x5582344e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.316+0000 7f633b400640 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5582344dfee0 0x5582344e22f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f9df8b8fa268b2 server_cookie=81b849034ca3f478 in_seq=0 out_seq=0 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.316+0000 7f6324c00640 10 client.25519 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.317+0000 7f6324c00640 1 -- 172.21.15.12:0/2078583239 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558234437f90 con 0x5582344dfee0 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.317+0000 7f6324c00640 10 client.25519 handle_command_reply: tid=0 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.317+0000 7f634461f740 1 client.25519 shutdown 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.317+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 msgr2=0x5582344cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.317+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cd5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5582344c2680 tx=0x7f6328012b60 comp rx=0 tx=0).stop 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 shutdown_connections 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5582344dfee0 0x5582344e22f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344cd5d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 --2- 172.21.15.12:0/2078583239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f6324c00640 0 client.25519 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 >> 172.21.15.12:0/2078583239 conn(0x5582344c14f0 msgr2=0x5582344c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 shutdown_connections 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 1 -- 172.21.15.12:0/2078583239 wait complete. 2024-04-26T22:04:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.318+0000 7f634461f740 20 client.25519 trim_cache size 0 max 16384 2024-04-26T22:04:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.336+0000 7f634461f740 1 Processor -- start 2024-04-26T22:04:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.336+0000 7f634461f740 1 -- start start 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c5900 0x5582344e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344d01c0 0x5582344e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1afd10 con 0x5582344c5900 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1b2590 con 0x5582344c4fe0 2024-04-26T22:04:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f634461f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1b0380 con 0x5582344d01c0 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f633b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f633b400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38456/0 (socket says 172.21.15.12:38456) 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f633b400640 1 -- 172.21.15.12:0/1079282584 learned_addr learned my addr 172.21.15.12:0/1079282584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f633a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c5900 0x5582344e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.337+0000 7f633b400640 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344d01c0 0x5582344e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 -- 172.21.15.12:0/1079282584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344d01c0 msgr2=0x5582344e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344d01c0 0x5582344e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 -- 172.21.15.12:0/1079282584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c5900 msgr2=0x5582344e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c5900 0x5582344e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 -- 172.21.15.12:0/1079282584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558234208620 con 0x5582344c4fe0 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f633b400640 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6334004b00 tx=0x7f63340096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2158872370be9e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f6327400640 1 -- 172.21.15.12:0/1079282584 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334004050 con 0x5582344c4fe0 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f6327400640 1 -- 172.21.15.12:0/1079282584 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633400f480 con 0x5582344c4fe0 2024-04-26T22:04:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.338+0000 7f6327400640 1 -- 172.21.15.12:0/1079282584 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334019690 con 0x5582344c4fe0 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- 172.21.15.12:0/1079282584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 msgr2=0x5582344dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6334004b00 tx=0x7f63340096c0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- 172.21.15.12:0/1079282584 shutdown_connections 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344d01c0 0x5582344e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c5900 0x5582344e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 --2- 172.21.15.12:0/1079282584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344c4fe0 0x5582344dbcb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- 172.21.15.12:0/1079282584 >> 172.21.15.12:0/1079282584 conn(0x5582344c14f0 msgr2=0x5582344c8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- 172.21.15.12:0/1079282584 shutdown_connections 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- 172.21.15.12:0/1079282584 wait complete. 2024-04-26T22:04:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.339+0000 7f634461f740 1 -- start start 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 Processor -- start 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1b0a50 con 0x5582344c4fe0 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1b1ec0 con 0x5582344d01c0 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1b1120 con 0x5582344c5900 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f634461f740 10 client.0 _ll_register_callbacks cb 0x7f6328009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f633b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40302/0 (socket says 172.21.15.12:40302) 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.340+0000 7f633b400640 1 -- 172.21.15.12:0/1445769561 learned_addr learned my addr 172.21.15.12:0/1445769561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633a000640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 -- 172.21.15.12:0/1445769561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 msgr2=0x5582344c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 -- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 msgr2=0x5582344c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 -- 172.21.15.12:0/1445769561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582343e4cb0 con 0x5582344c4fe0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633a000640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f633b400640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6334006010 tx=0x7f6334009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f83dfcf57dd027 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f6324c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633402c070 con 0x5582344c4fe0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f6334003a30 con 0x5582344c4fe0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.341+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633400f480 con 0x5582344c4fe0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6334046050 con 0x5582344c4fe0 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 10 client.15408 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55823454e8d0 con 0x5582344c4fe0 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633400f480 con 0x5582344c4fe0 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6334041030 con 0x5582344c4fe0 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 10 client.15408 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 20 client.15408 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55823454ed80 0x5582345511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 4 client.15408 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.342+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558234431e00 con 0x55823454ed80 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.343+0000 7f633a000640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55823454ed80 0x5582345511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.343+0000 7f633a000640 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55823454ed80 0x5582345511a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80ce65d2d9fee0b server_cookie=8bf01fe753bf0a81 in_seq=0 out_seq=0 2024-04-26T22:04:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.343+0000 7f6324c00640 10 client.15408 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f6324c00640 1 -- 172.21.15.12:0/1445769561 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558234431e00 con 0x55823454ed80 2024-04-26T22:04:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f6324c00640 10 client.15408 handle_command_reply: tid=0 2024-04-26T22:04:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 client.15408 shutdown 2024-04-26T22:04:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 msgr2=0x5582344d83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6334006010 tx=0x7f6334009ed0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 shutdown_connections 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55823454ed80 0x5582345511a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582344d01c0 0x5582344c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582344c5900 0x5582344c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 --2- 172.21.15.12:0/1445769561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5582344c4fe0 0x5582344d83b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f6324c00640 0 client.15408 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 >> 172.21.15.12:0/1445769561 conn(0x5582344c14f0 msgr2=0x5582344c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 shutdown_connections 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.350+0000 7f634461f740 1 -- 172.21.15.12:0/1445769561 wait complete. 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f634461f740 20 client.15408 trim_cache size 0 max 16384 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:04:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 msgr2=0x7f63200a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 0x7f63200a6250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x5582344c6580 tx=0x7f632c0f4940 comp rx=0 tx=0).stop 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 msgr2=0x7f633c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f632c005ab0 tx=0x7f632c003c10 comp rx=0 tx=0).stop 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 shutdown_connections 2024-04-26T22:04:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f63200a3d90 0x7f63200a6250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f633c10cd90 0x7f633c19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f633c104240 0x7f633c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 --2- 172.21.15.12:0/2386816388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f633c103920 0x7f633c19b160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 >> 172.21.15.12:0/2386816388 conn(0x7f633c0fd180 msgr2=0x7f633c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.351+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 shutdown_connections 2024-04-26T22:04:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:11.352+0000 7f633be00640 1 -- 172.21.15.12:0/2386816388 wait complete. 2024-04-26T22:04:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:11 smithi012 ceph-mon[28184]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:11.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:11 smithi154 ceph-mon[32578]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:11 smithi077 ceph-mon[32280]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:12.073 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T22:04:12.203 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T22:04:12.291 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T22:04:12.297 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T22:04:12.297 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T22:04:12.383 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T22:04:12.394 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:04:12.394 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T22:04:12.448 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-26T22:04:12.449 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-26T22:04:12.449 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-26T22:04:12.524 INFO:tasks.workunit.client.0.smithi012.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Iozone: Performance Test of File I/O 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Version $Revision: 3.494 $ 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Compiled for 64 bit mode. 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Build: linux-AMD64 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T22:04:12.636 INFO:tasks.workunit.client.0.smithi012.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Run began: Fri Apr 26 22:04:12 2024 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Include close in write timing 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Include fsync in write timing 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: File size set to 1048576 kB 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Record Size 16 kB 2024-04-26T22:04:12.637 INFO:tasks.workunit.client.0.smithi012.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Output is in kBytes/sec 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache line size set to 32 bytes. 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: File stride size set to 17 * record size. 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Throughput test with 1 process 2024-04-26T22:04:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-26T22:04:12.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:12 smithi154 ceph-mon[32578]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:12.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:12 smithi077 ceph-mon[32280]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:13.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:12 smithi012 ceph-mon[28184]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T22:04:15.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:15 smithi012 ceph-mon[28184]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T22:04:15.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:15 smithi154 ceph-mon[32578]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T22:04:15.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:15 smithi077 ceph-mon[32280]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 initial writers = 239480.53 kB/sec 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 initial writers = 239445.03 kB/sec 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 239480.53 kB/sec 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 239480.53 kB/sec 2024-04-26T22:04:17.021 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 239480.53 kB/sec 2024-04-26T22:04:17.022 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:17.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:17 smithi012 ceph-mon[28184]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T22:04:17.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:17 smithi154 ceph-mon[32578]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T22:04:17.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:17 smithi077 ceph-mon[32280]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 938 B/s wr, 2 op/s 2024-04-26T22:04:19.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:19 smithi012 ceph-mon[28184]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:19.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:19 smithi154 ceph-mon[32578]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:19.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:19 smithi077 ceph-mon[32280]: pgmap v54: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:21.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:21 smithi012 ceph-mon[28184]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:21.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:21 smithi154 ceph-mon[32578]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:21.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:21 smithi077 ceph-mon[32280]: pgmap v55: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:23.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:23 smithi012 ceph-mon[28184]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:23.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:23 smithi154 ceph-mon[32578]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:23.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:23 smithi077 ceph-mon[32280]: pgmap v56: 129 pgs: 129 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T22:04:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.580+0000 7f21af800640 1 -- 172.21.15.12:0/4113313706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 msgr2=0x7f21a810afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.580+0000 7f21af800640 1 --2- 172.21.15.12:0/4113313706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810afd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f219c009930 tx=0x7f219c02a420 comp rx=0 tx=0).stop 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.581+0000 7f21af800640 1 -- 172.21.15.12:0/4113313706 shutdown_connections 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.581+0000 7f21af800640 1 --2- 172.21.15.12:0/4113313706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 0x7f21a81119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.581+0000 7f21af800640 1 --2- 172.21.15.12:0/4113313706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 0x7f21a8104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.581+0000 7f21af800640 1 --2- 172.21.15.12:0/4113313706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810afd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.581+0000 7f21af800640 1 -- 172.21.15.12:0/4113313706 >> 172.21.15.12:0/4113313706 conn(0x7f21a8100620 msgr2=0x7f21a8102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.582+0000 7f21af800640 1 -- 172.21.15.12:0/4113313706 shutdown_connections 2024-04-26T22:04:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.583+0000 7f21af800640 1 -- 172.21.15.12:0/4113313706 wait complete. 2024-04-26T22:04:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.586+0000 7f21af800640 1 Processor -- start 2024-04-26T22:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.588+0000 7f21af800640 1 -- start start 2024-04-26T22:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 0x7f21a8111780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 0x7f21a810e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8114e30 con 0x7f21a8104b30 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8114cb0 con 0x7f21a810abf0 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21af800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8114fb0 con 0x7f21a81054f0 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.589+0000 7f21ad800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52666/0 (socket says 172.21.15.12:52666) 2024-04-26T22:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 -- 172.21.15.12:0/337004429 learned_addr learned my addr 172.21.15.12:0/337004429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ace00640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 0x7f21a8111780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21a7e00640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 0x7f21a810e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 -- 172.21.15.12:0/337004429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 msgr2=0x7f21a810e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 0x7f21a810e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 -- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 msgr2=0x7f21a8111780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 0x7f21a8111780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.590+0000 7f21ad800640 1 -- 172.21.15.12:0/337004429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a810f5c0 con 0x7f21a810abf0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21ad800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21a000eb10 tx=0x7f21a000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21a5600640 1 -- 172.21.15.12:0/337004429 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004b20 con 0x7f21a810abf0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21a5600640 1 -- 172.21.15.12:0/337004429 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a0004cc0 con 0x7f21a810abf0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a819a300 con 0x7f21a810abf0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21a5600640 1 -- 172.21.15.12:0/337004429 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0010720 con 0x7f21a810abf0 2024-04-26T22:04:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.591+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a819a870 con 0x7f21a810abf0 2024-04-26T22:04:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.596+0000 7f21a5600640 1 -- 172.21.15.12:0/337004429 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f21a0005020 con 0x7f21a810abf0 2024-04-26T22:04:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.598+0000 7f21a5600640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 0x7f21900a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.599+0000 7f21a5600640 1 -- 172.21.15.12:0/337004429 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f21a00d1110 con 0x7f21a810abf0 2024-04-26T22:04:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.599+0000 7f21ace00640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 0x7f21900a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.602+0000 7f21b25e7740 1 Processor -- start 2024-04-26T22:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 -- start start 2024-04-26T22:04:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21ace00640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 0x7f21900a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f219c031440 tx=0x7f219c005a20 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd76f0 0x56461fde1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 0x56461fde43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56461fd3c380 con 0x56461fde1fb0 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56461fcd8c30 con 0x56461fdd76f0 2024-04-26T22:04:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21b25e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56461fd38340 con 0x56461fdd6dd0 2024-04-26T22:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21ace00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21ace00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43236/0 (socket says 172.21.15.12:43236) 2024-04-26T22:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.603+0000 7f21ace00640 1 -- 172.21.15.12:0/3325767966 learned_addr learned my addr 172.21.15.12:0/3325767966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21a7e00640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd76f0 0x56461fde1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ad800640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 0x56461fde43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ace00640 1 -- 172.21.15.12:0/3325767966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd76f0 msgr2=0x56461fde1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ace00640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd76f0 0x56461fde1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ace00640 1 -- 172.21.15.12:0/3325767966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 msgr2=0x56461fde43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ace00640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 0x56461fde43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ace00640 1 -- 172.21.15.12:0/3325767966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461fcfa940 con 0x56461fdd6dd0 2024-04-26T22:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.604+0000 7f21ad800640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 0x56461fde43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.605+0000 7f21ace00640 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f219c0096c0 tx=0x7f219c03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2588e9fde9510593 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.605+0000 7f219a000640 1 -- 172.21.15.12:0/3325767966 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c035040 con 0x56461fdd6dd0 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.605+0000 7f219a000640 1 -- 172.21.15.12:0/3325767966 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c04e070 con 0x56461fdd6dd0 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.605+0000 7f219a000640 1 -- 172.21.15.12:0/3325767966 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c003d80 con 0x56461fdd6dd0 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 -- 172.21.15.12:0/3325767966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 msgr2=0x56461fdd71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f219c0096c0 tx=0x7f219c03d000 comp rx=0 tx=0).stop 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 -- 172.21.15.12:0/3325767966 shutdown_connections 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fde1fb0 0x56461fde43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd76f0 0x56461fde1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3325767966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd6dd0 0x56461fdd71b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.606+0000 7f21b25e7740 1 -- 172.21.15.12:0/3325767966 >> 172.21.15.12:0/3325767966 conn(0x56461fdd32e0 msgr2=0x56461fdd36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.607+0000 7f21b25e7740 1 -- 172.21.15.12:0/3325767966 shutdown_connections 2024-04-26T22:04:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.607+0000 7f21b25e7740 1 -- 172.21.15.12:0/3325767966 wait complete. 2024-04-26T22:04:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.607+0000 7f21b25e7740 1 -- start start 2024-04-26T22:04:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 Processor -- start 2024-04-26T22:04:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fddef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fddf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56461fde4b80 con 0x56461fdd6dd0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56461fcc3c60 con 0x56461fde1fb0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56461fde4d00 con 0x56461fdd76f0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21b25e7740 10 client.0 _ll_register_callbacks cb 0x56461fcefe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.608+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43252/0 (socket says 172.21.15.12:43252) 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 -- 172.21.15.12:0/3114941641 learned_addr learned my addr 172.21.15.12:0/3114941641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21ad800640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fddf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21ace00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fddef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 -- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 msgr2=0x56461fddf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fddf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 -- 172.21.15.12:0/3114941641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 msgr2=0x56461fddef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fddef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21a7e00640 1 -- 172.21.15.12:0/3114941641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461fcd2510 con 0x56461fdd76f0 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21ad800640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fddf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.609+0000 7f21ace00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fddef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.610+0000 7f21a7e00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2194012700 tx=0x7f2194012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af97e97f7ead7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.610+0000 7f218ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.610+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194004320 con 0x56461fdd76f0 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.610+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f21940119a0 con 0x56461fdd76f0 2024-04-26T22:04:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.610+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f219c04e070 con 0x56461fdd76f0 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.611+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21940044c0 con 0x56461fdd76f0 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.611+0000 7f21b25e7740 10 client.25534 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.611+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56461fa4e210 con 0x56461fdd76f0 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.611+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c04e070 con 0x56461fdd76f0 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.611+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f219400f440 con 0x56461fdd76f0 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 10 client.25534 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 20 client.25534 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fdf1cd0 0x56461fdf4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 4 client.25534 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56461fd49f90 con 0x56461fdf1cd0 2024-04-26T22:04:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.612+0000 7f21ace00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fdf1cd0 0x56461fdf4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.613+0000 7f21ace00640 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fdf1cd0 0x56461fdf4100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8f62a2feee5b54 server_cookie=4eb6d866d1989a1d in_seq=0 out_seq=0 2024-04-26T22:04:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.613+0000 7f218ea00640 10 client.25534 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.614+0000 7f218ea00640 1 -- 172.21.15.12:0/3114941641 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56461fd49f90 con 0x56461fdf1cd0 2024-04-26T22:04:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.614+0000 7f218ea00640 10 client.25534 handle_command_reply: tid=0 2024-04-26T22:04:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.615+0000 7f21b25e7740 1 client.25534 shutdown 2024-04-26T22:04:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.616+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 msgr2=0x56461fddf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2194012700 tx=0x7f2194012bc0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 shutdown_connections 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fdf1cd0 0x56461fdf4100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fddf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fddf450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f21b25e7740 1 --2- 172.21.15.12:0/3114941641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fddef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.617+0000 7f218ea00640 0 client.25534 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.618+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 >> 172.21.15.12:0/3114941641 conn(0x56461fdd32e0 msgr2=0x56461fdd9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.620+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 shutdown_connections 2024-04-26T22:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.620+0000 7f21b25e7740 1 -- 172.21.15.12:0/3114941641 wait complete. 2024-04-26T22:04:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.620+0000 7f21b25e7740 20 client.25534 trim_cache size 0 max 16384 2024-04-26T22:04:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.650+0000 7f21b25e7740 1 Processor -- start 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.651+0000 7f21b25e7740 1 -- start start 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 0x56461fdeda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 0x56461fddf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81c0060 con 0x56461fdd76f0 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81bf2c0 con 0x56461fdd6dd0 2024-04-26T22:04:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.652+0000 7f21b25e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81a2720 con 0x56461fde1fb0 2024-04-26T22:04:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.653+0000 7f21ace00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.653+0000 7f21ace00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:52704/0 (socket says 172.21.15.12:52704) 2024-04-26T22:04:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.653+0000 7f21ace00640 1 -- 172.21.15.12:0/180838515 learned_addr learned my addr 172.21.15.12:0/180838515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.654+0000 7f21ad800640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 0x56461fddf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.654+0000 7f21a7e00640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 0x56461fdeda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.654+0000 7f21ace00640 1 -- 172.21.15.12:0/180838515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 msgr2=0x56461fddf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.657+0000 7f21ace00640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 0x56461fddf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.657+0000 7f21ace00640 1 -- 172.21.15.12:0/180838515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 msgr2=0x56461fdeda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.657+0000 7f21ace00640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 0x56461fdeda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.657+0000 7f21ace00640 1 -- 172.21.15.12:0/180838515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461fb1a620 con 0x56461fdd6dd0 2024-04-26T22:04:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.658+0000 7f21a7e00640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 0x56461fdeda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.658+0000 7f21ad800640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 0x56461fddf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.658+0000 7f21ace00640 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f219c004650 tx=0x7f219c02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fe14f5562c691f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.658+0000 7f2199600640 1 -- 172.21.15.12:0/180838515 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c062070 con 0x56461fdd6dd0 2024-04-26T22:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.659+0000 7f2199600640 1 -- 172.21.15.12:0/180838515 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c04e050 con 0x56461fdd6dd0 2024-04-26T22:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.659+0000 7f2199600640 1 -- 172.21.15.12:0/180838515 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c035040 con 0x56461fdd6dd0 2024-04-26T22:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- 172.21.15.12:0/180838515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 msgr2=0x56461fde1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f219c004650 tx=0x7f219c02a420 comp rx=0 tx=0).stop 2024-04-26T22:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- 172.21.15.12:0/180838515 shutdown_connections 2024-04-26T22:04:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fde1fb0 0x56461fddf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd76f0 0x56461fdeda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 --2- 172.21.15.12:0/180838515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fdd6dd0 0x56461fde1640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- 172.21.15.12:0/180838515 >> 172.21.15.12:0/180838515 conn(0x56461fdd32e0 msgr2=0x56461fddeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- 172.21.15.12:0/180838515 shutdown_connections 2024-04-26T22:04:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- 172.21.15.12:0/180838515 wait complete. 2024-04-26T22:04:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.660+0000 7f21b25e7740 1 -- start start 2024-04-26T22:04:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.661+0000 7f21b25e7740 1 Processor -- start 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.661+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fde5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fe57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81bd780 con 0x56461fdd6dd0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81bebf0 con 0x56461fde1fb0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21a81bde50 con 0x56461fdd76f0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21b25e7740 10 client.0 _ll_register_callbacks cb 0x7f21940097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43268/0 (socket says 172.21.15.12:43268) 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 -- 172.21.15.12:0/1746702725 learned_addr learned my addr 172.21.15.12:0/1746702725 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21ace00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fde5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21ad800640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fe57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 -- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 msgr2=0x56461fe57280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fe57280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 -- 172.21.15.12:0/1746702725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 msgr2=0x56461fde5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fde5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21a7e00640 1 -- 172.21.15.12:0/1746702725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461fcf6cb0 con 0x56461fdd76f0 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.662+0000 7f21ace00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fde5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f21a7e00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f219400fa00 tx=0x7f2194005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e146a9f7476a3f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f218ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219402c040 con 0x56461fdd76f0 2024-04-26T22:04:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f219400f440 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f219c04e050 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f219400fe50 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f21b25e7740 10 client.25537 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.663+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56461fe58740 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.664+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c04e050 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2194038030 con 0x56461fdd76f0 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 10 client.25537 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 20 client.25537 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fe58ee0 0x56461fe5b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 4 client.25537 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56461fd43e00 con 0x56461fe58ee0 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.665+0000 7f21ace00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fe58ee0 0x56461fe5b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.666+0000 7f21ace00640 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fe58ee0 0x56461fe5b300 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83bbb4a221b36f7f server_cookie=20ed1303d542ee95 in_seq=0 out_seq=0 2024-04-26T22:04:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.666+0000 7f218ea00640 10 client.25537 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.687+0000 7f218ea00640 1 -- 172.21.15.12:0/1746702725 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56461fd43e00 con 0x56461fe58ee0 2024-04-26T22:04:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.687+0000 7f218ea00640 10 client.25537 handle_command_reply: tid=0 2024-04-26T22:04:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.687+0000 7f21b25e7740 1 client.25537 shutdown 2024-04-26T22:04:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.687+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 msgr2=0x56461fde6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.687+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f219400fa00 tx=0x7f2194005670 comp rx=0 tx=0).stop 2024-04-26T22:04:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 shutdown_connections 2024-04-26T22:04:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x56461fe58ee0 0x56461fe5b300 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fde1fb0 0x56461fe57280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56461fdd76f0 0x56461fde6220 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 --2- 172.21.15.12:0/1746702725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56461fdd6dd0 0x56461fde5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 >> 172.21.15.12:0/1746702725 conn(0x56461fdd32e0 msgr2=0x56461fddf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.688+0000 7f218ea00640 0 client.25537 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.689+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 shutdown_connections 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.689+0000 7f21b25e7740 1 -- 172.21.15.12:0/1746702725 wait complete. 2024-04-26T22:04:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.689+0000 7f21b25e7740 20 client.25537 trim_cache size 0 max 16384 2024-04-26T22:04:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:04:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:04:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:04:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.691+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 msgr2=0x7f21900a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.691+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 0x7f21900a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f219c031440 tx=0x7f219c005a20 comp rx=0 tx=0).stop 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.691+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 msgr2=0x7f21a810edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.691+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21a000eb10 tx=0x7f21a000c2d0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 shutdown_connections 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f21900a3ec0 0x7f21900a6380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21a810abf0 0x7f21a810edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21a81054f0 0x7f21a810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 --2- 172.21.15.12:0/337004429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21a8104b30 0x7f21a8111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 >> 172.21.15.12:0/337004429 conn(0x7f21a8100620 msgr2=0x7f21a8101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 shutdown_connections 2024-04-26T22:04:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:24.692+0000 7f21af800640 1 -- 172.21.15.12:0/337004429 wait complete. 2024-04-26T22:04:24.832 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:24.832 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 rewriters = 250710.48 kB/sec 2024-04-26T22:04:24.832 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 rewriters = 250671.87 kB/sec 2024-04-26T22:04:24.833 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 250710.48 kB/sec 2024-04-26T22:04:24.833 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 250710.48 kB/sec 2024-04-26T22:04:24.833 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 250710.48 kB/sec 2024-04-26T22:04:24.833 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:25.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:25 smithi154 ceph-mon[32578]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 105 MiB/s wr, 31 op/s 2024-04-26T22:04:25.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:25 smithi077 ceph-mon[32280]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 105 MiB/s wr, 31 op/s 2024-04-26T22:04:25.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:25 smithi012 ceph-mon[28184]: pgmap v57: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 105 MiB/s wr, 31 op/s 2024-04-26T22:04:27.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:27 smithi154 ceph-mon[32578]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 105 MiB/s wr, 29 op/s 2024-04-26T22:04:27.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:27 smithi077 ceph-mon[32280]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 105 MiB/s wr, 29 op/s 2024-04-26T22:04:27.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:27 smithi012 ceph-mon[28184]: pgmap v58: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 105 MiB/s wr, 29 op/s 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 readers = 3553819.75 kB/sec 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 readers = 3545697.08 kB/sec 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 3553819.75 kB/sec 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 3553819.75 kB/sec 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 3553819.75 kB/sec 2024-04-26T22:04:27.969 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:29.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:29 smithi154 ceph-mon[32578]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 50 op/s 2024-04-26T22:04:29.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:29 smithi077 ceph-mon[32280]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 50 op/s 2024-04-26T22:04:29.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:29 smithi012 ceph-mon[28184]: pgmap v59: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 50 op/s 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 re-readers = 2607509.50 kB/sec 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 re-readers = 2602958.40 kB/sec 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 2607509.50 kB/sec 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 2607509.50 kB/sec 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 2607509.50 kB/sec 2024-04-26T22:04:30.457 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:31.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:31 smithi154 ceph-mon[32578]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:31.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:31 smithi077 ceph-mon[32280]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:31.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:31 smithi012 ceph-mon[28184]: pgmap v60: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:33.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:33 smithi154 ceph-mon[32578]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:33.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:33 smithi077 ceph-mon[32280]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:33.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:33 smithi012 ceph-mon[28184]: pgmap v61: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T22:04:35.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:35 smithi154 ceph-mon[32578]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:35.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:35 smithi077 ceph-mon[32280]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:35 smithi012 ceph-mon[28184]: pgmap v62: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:37.076 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.076 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.076 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.076 INFO:tasks.workunit.client.0.smithi012.stdout:iozone test complete. 2024-04-26T22:04:37.082 INFO:tasks.workunit.client.0.smithi012.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Iozone: Performance Test of File I/O 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Version $Revision: 3.494 $ 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Compiled for 64 bit mode. 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Build: linux-AMD64 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T22:04:37.175 INFO:tasks.workunit.client.0.smithi012.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Run began: Fri Apr 26 22:04:37 2024 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Include close in write timing 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Include fsync in write timing 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: File size set to 1048576 kB 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Record Size 1024 kB 2024-04-26T22:04:37.176 INFO:tasks.workunit.client.0.smithi012.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Output is in kBytes/sec 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache line size set to 32 bytes. 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: File stride size set to 17 * record size. 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Throughput test with 1 process 2024-04-26T22:04:37.177 INFO:tasks.workunit.client.0.smithi012.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-26T22:04:37.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:37 smithi154 ceph-mon[32578]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 66 MiB/s wr, 21 op/s 2024-04-26T22:04:37.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:37 smithi077 ceph-mon[32280]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 66 MiB/s wr, 21 op/s 2024-04-26T22:04:37.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:37 smithi012 ceph-mon[28184]: pgmap v63: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 66 MiB/s wr, 21 op/s 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 -- 172.21.15.12:0/2238128607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8581137c0 msgr2=0x7fa858115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 --2- 172.21.15.12:0/2238128607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8581137c0 0x7fa858115bb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa850009be0 tx=0x7fa85002a6c0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 -- 172.21.15.12:0/2238128607 shutdown_connections 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 --2- 172.21.15.12:0/2238128607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8581137c0 0x7fa858115bb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 --2- 172.21.15.12:0/2238128607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa858077780 0x7fa858113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 --2- 172.21.15.12:0/2238128607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858076e60 0x7fa858077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 -- 172.21.15.12:0/2238128607 >> 172.21.15.12:0/2238128607 conn(0x7fa858100630 msgr2=0x7fa858102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 -- 172.21.15.12:0/2238128607 shutdown_connections 2024-04-26T22:04:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.852+0000 7fa85f400640 1 -- 172.21.15.12:0/2238128607 wait complete. 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 Processor -- start 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 -- start start 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581137c0 0x7fa8581a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa858119190 con 0x7fa858076e60 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa858119010 con 0x7fa858077780 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.853+0000 7fa85f400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa858119310 con 0x7fa8581137c0 2024-04-26T22:04:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa85ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60900/0 (socket says 172.21.15.12:60900) 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa85ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35164/0 (socket says 172.21.15.12:35164) 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 -- 172.21.15.12:0/2017217159 learned_addr learned my addr 172.21.15.12:0/2017217159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa85d400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581137c0 0x7fa8581a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 -- 172.21.15.12:0/2017217159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581137c0 msgr2=0x7fa8581a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581137c0 0x7fa8581a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 -- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 msgr2=0x7fa85819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa857e00640 1 -- 172.21.15.12:0/2017217159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a4610 con 0x7fa858077780 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.854+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa857e00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9c0 tx=0x7fa84400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa855600640 1 -- 172.21.15.12:0/2017217159 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84400cde0 con 0x7fa858077780 2024-04-26T22:04:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a4900 con 0x7fa858077780 2024-04-26T22:04:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa855600640 1 -- 172.21.15.12:0/2017217159 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8440035f0 con 0x7fa858077780 2024-04-26T22:04:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa855600640 1 -- 172.21.15.12:0/2017217159 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844010690 con 0x7fa858077780 2024-04-26T22:04:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.855+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581ac190 con 0x7fa858077780 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.858+0000 7fa8623d1740 1 Processor -- start 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.858+0000 7fa8623d1740 1 -- start start 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.858+0000 7fa855600640 1 -- 172.21.15.12:0/2017217159 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa844004000 con 0x7fa858077780 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c92a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fb60bed380 con 0x55fb60c886f0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fb60b89c30 con 0x55fb60c87dd0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa8623d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb60be9340 con 0x55fb60c92fb0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c92a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60912/0 (socket says 172.21.15.12:60912) 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 -- 172.21.15.12:0/584917629 learned_addr learned my addr 172.21.15.12:0/584917629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa857e00640 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 -- 172.21.15.12:0/584917629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 msgr2=0x55fb60c953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 -- 172.21.15.12:0/584917629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 msgr2=0x55fb60c92a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c92a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.859+0000 7fa85d400640 1 -- 172.21.15.12:0/584917629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb60bab940 con 0x55fb60c87dd0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.860+0000 7fa85d400640 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa85806be30 tx=0x7fa850030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523b6700aae26297 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.860+0000 7fa849c00640 1 -- 172.21.15.12:0/584917629 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850041030 con 0x55fb60c87dd0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.860+0000 7fa849c00640 1 -- 172.21.15.12:0/584917629 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa850003e20 con 0x55fb60c87dd0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.860+0000 7fa849c00640 1 -- 172.21.15.12:0/584917629 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850036040 con 0x55fb60c87dd0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa855600640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 0x7fa8400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa855600640 1 -- 172.21.15.12:0/2017217159 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa8440d0eb0 con 0x7fa858077780 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 -- 172.21.15.12:0/584917629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 msgr2=0x55fb60c881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa85806be30 tx=0x7fa850030430 comp rx=0 tx=0).stop 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 0x7fa8400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 -- 172.21.15.12:0/584917629 shutdown_connections 2024-04-26T22:04:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c92a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 --2- 172.21.15.12:0/584917629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c881b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.861+0000 7fa8623d1740 1 -- 172.21.15.12:0/584917629 >> 172.21.15.12:0/584917629 conn(0x55fb60c842e0 msgr2=0x55fb60c846c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.862+0000 7fa8623d1740 1 -- 172.21.15.12:0/584917629 shutdown_connections 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.862+0000 7fa8623d1740 1 -- 172.21.15.12:0/584917629 wait complete. 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.862+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 0x7fa8400a6280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55fb60c89370 tx=0x7fa84c005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.862+0000 7fa8623d1740 1 -- start start 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 Processor -- start 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c8fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fb60c95b80 con 0x55fb60c886f0 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fb60b74c60 con 0x55fb60c87dd0 2024-04-26T22:04:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb60c95d00 con 0x55fb60c92fb0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa8623d1740 10 client.0 _ll_register_callbacks cb 0x55fb60ba0e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.863+0000 7fa857e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35174/0 (socket says 172.21.15.12:35174) 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 -- 172.21.15.12:0/2590813108 learned_addr learned my addr 172.21.15.12:0/2590813108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c8fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa85d400640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 -- 172.21.15.12:0/2590813108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 msgr2=0x55fb60c90950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c90950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 -- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 msgr2=0x55fb60c8fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c8fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 -- 172.21.15.12:0/2590813108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb60b83510 con 0x55fb60c886f0 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa85d400640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa857e00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa84400c6a0 tx=0x7fa844098da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4a599f375af8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.864+0000 7fa83e000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440042f0 con 0x55fb60c886f0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa844003aa0 con 0x55fb60c886f0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa850003e20 con 0x55fb60c886f0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa84409d140 con 0x55fb60c886f0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa8623d1740 10 client.15435 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fb608ff210 con 0x55fb60c886f0 2024-04-26T22:04:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa850003e20 con 0x55fb60c886f0 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.865+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa8440ce470 con 0x55fb60c886f0 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 10 client.15435 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 20 client.15435 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60ca2ce0 0x55fb60ca50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 4 client.15435 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb60bfaf90 con 0x55fb60ca2ce0 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.866+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60ca2ce0 0x55fb60ca50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.867+0000 7fa83e000640 10 client.15435 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.867+0000 7fa85ca00640 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60ca2ce0 0x55fb60ca50d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c4862d1b788a75 server_cookie=d2d4b274c8d0dd92 in_seq=0 out_seq=0 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa83e000640 1 -- 172.21.15.12:0/2590813108 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb60bfaf90 con 0x55fb60ca2ce0 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa83e000640 10 client.15435 handle_command_reply: tid=0 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 client.15435 shutdown 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 msgr2=0x55fb60c90410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa84400c6a0 tx=0x7fa844098da0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 shutdown_connections 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60ca2ce0 0x55fb60ca50d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c92fb0 0x55fb60c90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c90410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 --2- 172.21.15.12:0/2590813108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c87dd0 0x55fb60c8fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa83e000640 0 client.15435 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 >> 172.21.15.12:0/2590813108 conn(0x55fb60c842e0 msgr2=0x55fb60c8a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 shutdown_connections 2024-04-26T22:04:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 1 -- 172.21.15.12:0/2590813108 wait complete. 2024-04-26T22:04:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.868+0000 7fa8623d1740 20 client.15435 trim_cache size 0 max 16384 2024-04-26T22:04:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 Processor -- start 2024-04-26T22:04:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 -- start start 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 0x55fb60c8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b41e0 con 0x55fb60c92fb0 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b6a60 con 0x55fb60c886f0 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa8623d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b4850 con 0x55fb60c87dd0 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa85d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa857e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 0x55fb60c8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa85d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35188/0 (socket says 172.21.15.12:35188) 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa85d400640 1 -- 172.21.15.12:0/1516649111 learned_addr learned my addr 172.21.15.12:0/1516649111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.885+0000 7fa85ca00640 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 -- 172.21.15.12:0/1516649111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 msgr2=0x55fb60c9ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 -- 172.21.15.12:0/1516649111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 msgr2=0x55fb60c91d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 -- 172.21.15.12:0/1516649111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb609cb620 con 0x55fb60c886f0 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa85d400640 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa857e00640 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 0x55fb60c8f030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa84409c9d0 tx=0x7fa84409af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68310e4b38b8e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa849200640 1 -- 172.21.15.12:0/1516649111 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440042f0 con 0x55fb60c886f0 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa849200640 1 -- 172.21.15.12:0/1516649111 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8440a0df0 con 0x55fb60c886f0 2024-04-26T22:04:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.886+0000 7fa849200640 1 -- 172.21.15.12:0/1516649111 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84409c580 con 0x55fb60c886f0 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 -- 172.21.15.12:0/1516649111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 msgr2=0x55fb60c8f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 0x55fb60c8f030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa84409c9d0 tx=0x7fa84409af50 comp rx=0 tx=0).stop 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 -- 172.21.15.12:0/1516649111 shutdown_connections 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c92fb0 0x55fb60c91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c886f0 0x55fb60c8f030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 --2- 172.21.15.12:0/1516649111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 -- 172.21.15.12:0/1516649111 >> 172.21.15.12:0/1516649111 conn(0x55fb60c842e0 msgr2=0x55fb60c915f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 -- 172.21.15.12:0/1516649111 shutdown_connections 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.887+0000 7fa8623d1740 1 -- 172.21.15.12:0/1516649111 wait complete. 2024-04-26T22:04:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 -- start start 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 Processor -- start 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c92fb0 0x55fb60c96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b4f20 con 0x55fb60c886f0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b6390 con 0x55fb60c92fb0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.888+0000 7fa8623d1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b55f0 con 0x55fb60c87dd0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa8623d1740 10 client.0 _ll_register_callbacks cb 0x7fa8440985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35198/0 (socket says 172.21.15.12:35198) 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 -- 172.21.15.12:0/3474440623 learned_addr learned my addr 172.21.15.12:0/3474440623 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa85ca00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa85d400640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c92fb0 0x55fb60c96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 -- 172.21.15.12:0/3474440623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 msgr2=0x55fb60c9b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 -- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c92fb0 msgr2=0x55fb60c96f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c92fb0 0x55fb60c96f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 -- 172.21.15.12:0/3474440623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb60ba7cb0 con 0x55fb60c886f0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa857e00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa84409c740 tx=0x7fa844098670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c17ea28fbe769c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.889+0000 7fa83e000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440042f0 con 0x55fb60c886f0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa8440ce470 con 0x55fb60c886f0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8440a0df0 con 0x55fb60c886f0 2024-04-26T22:04:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa8440ce670 con 0x55fb60c886f0 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 10 client.15441 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fb60d09120 con 0x55fb60c886f0 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8440a0df0 con 0x55fb60c886f0 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa8440a1f80 con 0x55fb60c886f0 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 10 client.15441 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.890+0000 7fa8623d1740 20 client.15441 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60d095d0 0x55fb60d0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa8623d1740 4 client.15441 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb60bf4e00 con 0x55fb60d095d0 2024-04-26T22:04:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa85ca00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60d095d0 0x55fb60d0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa85ca00640 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60d095d0 0x55fb60d0b9f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edb48da41ae229 server_cookie=52b9500e3d89eb25 in_seq=0 out_seq=0 2024-04-26T22:04:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.891+0000 7fa83e000640 10 client.15441 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa83e000640 1 -- 172.21.15.12:0/3474440623 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb60bf4e00 con 0x55fb60d095d0 2024-04-26T22:04:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa83e000640 10 client.15441 handle_command_reply: tid=0 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 client.15441 shutdown 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 msgr2=0x55fb60c967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa84409c740 tx=0x7fa844098670 comp rx=0 tx=0).stop 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 shutdown_connections 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55fb60d095d0 0x55fb60d0b9f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fb60c92fb0 0x55fb60c96f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa83e000640 0 client.15441 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fb60c886f0 0x55fb60c967c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 --2- 172.21.15.12:0/3474440623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb60c87dd0 0x55fb60c9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 >> 172.21.15.12:0/3474440623 conn(0x55fb60c842e0 msgr2=0x55fb60c8a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.903+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 shutdown_connections 2024-04-26T22:04:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa8623d1740 1 -- 172.21.15.12:0/3474440623 wait complete. 2024-04-26T22:04:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa8623d1740 20 client.15441 trim_cache size 0 max 16384 2024-04-26T22:04:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:04:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:04:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:04:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 msgr2=0x7fa8400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 0x7fa8400a6280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55fb60c89370 tx=0x7fa84c005f90 comp rx=0 tx=0).stop 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 msgr2=0x7fa85819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9c0 tx=0x7fa84400ee80 comp rx=0 tx=0).stop 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 shutdown_connections 2024-04-26T22:04:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa8400a3dc0 0x7fa8400a6280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8581137c0 0x7fa8581a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa858077780 0x7fa85819fbb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 --2- 172.21.15.12:0/2017217159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858076e60 0x7fa85819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.904+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 >> 172.21.15.12:0/2017217159 conn(0x7fa858100630 msgr2=0x7fa858114e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.905+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 shutdown_connections 2024-04-26T22:04:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:37.905+0000 7fa85f400640 1 -- 172.21.15.12:0/2017217159 wait complete. 2024-04-26T22:04:39.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:39 smithi154 ceph-mon[32578]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 66 MiB/s wr, 22 op/s 2024-04-26T22:04:39.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:39 smithi077 ceph-mon[32280]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 66 MiB/s wr, 22 op/s 2024-04-26T22:04:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:39 smithi012 ceph-mon[28184]: pgmap v64: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 66 MiB/s wr, 22 op/s 2024-04-26T22:04:41.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:41 smithi154 ceph-mon[32578]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 initial writers = 243097.39 kB/sec 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 initial writers = 243033.60 kB/sec 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 243097.39 kB/sec 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 243097.39 kB/sec 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 243097.39 kB/sec 2024-04-26T22:04:41.614 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:41.614 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:41 smithi077 ceph-mon[32280]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:41.615 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:41 smithi012 ceph-mon[28184]: pgmap v65: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:43.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:43 smithi154 ceph-mon[32578]: pgmap v66: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:43 smithi077 ceph-mon[32280]: pgmap v66: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:43.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:43 smithi012 ceph-mon[28184]: pgmap v66: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-04-26T22:04:45.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:45 smithi154 ceph-mon[32578]: pgmap v67: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-04-26T22:04:45.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:45 smithi077 ceph-mon[32280]: pgmap v67: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-04-26T22:04:45.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:45 smithi012 ceph-mon[28184]: pgmap v67: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-04-26T22:04:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:47 smithi077 ceph-mon[32280]: pgmap v68: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 85 MiB/s wr, 50 op/s 2024-04-26T22:04:47.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:47 smithi012 ceph-mon[28184]: pgmap v68: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 85 MiB/s wr, 50 op/s 2024-04-26T22:04:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:47 smithi154 ceph-mon[32578]: pgmap v68: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 85 MiB/s wr, 50 op/s 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 rewriters = 272185.44 kB/sec 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 rewriters = 272113.71 kB/sec 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 272185.44 kB/sec 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 272185.44 kB/sec 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 272185.44 kB/sec 2024-04-26T22:04:48.017 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:48.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:48 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:48.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:48 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:48.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:48 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:49 smithi077 ceph-mon[32280]: pgmap v69: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 147 MiB/s wr, 67 op/s 2024-04-26T22:04:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:49 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:49 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:49 smithi012 ceph-mon[28184]: pgmap v69: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 147 MiB/s wr, 67 op/s 2024-04-26T22:04:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:49 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:49 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:49 smithi154 ceph-mon[32578]: pgmap v69: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 147 MiB/s wr, 67 op/s 2024-04-26T22:04:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:49 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:49 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:50.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:50 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:04:50.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:50 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:04:50.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:50 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 readers = 5062599.50 kB/sec 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 readers = 5043458.93 kB/sec 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 5062599.50 kB/sec 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 5062599.50 kB/sec 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 5062599.50 kB/sec 2024-04-26T22:04:51.054 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 -- 172.21.15.12:0/2664365298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2541032e0 msgr2=0x7fb25410f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 --2- 172.21.15.12:0/2664365298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2541032e0 0x7fb25410f7f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009cd0 tx=0x7fb24c02a8f0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 -- 172.21.15.12:0/2664365298 shutdown_connections 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 --2- 172.21.15.12:0/2664365298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2541032e0 0x7fb25410f7f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 --2- 172.21.15.12:0/2664365298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb254102920 0x7fb254102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 --2- 172.21.15.12:0/2664365298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254108930 0x7fb254108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 -- 172.21.15.12:0/2664365298 >> 172.21.15.12:0/2664365298 conn(0x7fb2540fe3f0 msgr2=0x7fb254100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.062+0000 7fb25b600640 1 -- 172.21.15.12:0/2664365298 shutdown_connections 2024-04-26T22:04:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.063+0000 7fb25b600640 1 -- 172.21.15.12:0/2664365298 wait complete. 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 Processor -- start 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 -- start start 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 0x7fb25419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 0x7fb25419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb254112bf0 con 0x7fb254108930 2024-04-26T22:04:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb254112a70 con 0x7fb2541032e0 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.064+0000 7fb25b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb254112d70 con 0x7fb254102920 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58400/0 (socket says 172.21.15.12:58400) 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 -- 172.21.15.12:0/2376138880 learned_addr learned my addr 172.21.15.12:0/2376138880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb258c00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 0x7fb25419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb253e00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 0x7fb25419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 -- 172.21.15.12:0/2376138880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 msgr2=0x7fb25419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 0x7fb25419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 -- 172.21.15.12:0/2376138880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 msgr2=0x7fb25419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 0x7fb25419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 -- 172.21.15.12:0/2376138880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2541a0240 con 0x7fb254108930 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb258c00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 0x7fb25419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb253e00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 0x7fb25419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.065+0000 7fb259600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005e30 tx=0x7fb24c030f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.066+0000 7fb251600640 1 -- 172.21.15.12:0/2376138880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c004040 con 0x7fb254108930 2024-04-26T22:04:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.066+0000 7fb251600640 1 -- 172.21.15.12:0/2376138880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24c040070 con 0x7fb254108930 2024-04-26T22:04:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.066+0000 7fb251600640 1 -- 172.21.15.12:0/2376138880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c045ac0 con 0x7fb254108930 2024-04-26T22:04:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.066+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2541a04d0 con 0x7fb254108930 2024-04-26T22:04:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.066+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541a7dc0 con 0x7fb254108930 2024-04-26T22:04:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.069+0000 7fb251600640 1 -- 172.21.15.12:0/2376138880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb24c045c60 con 0x7fb254108930 2024-04-26T22:04:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.069+0000 7fb25e40b740 1 Processor -- start 2024-04-26T22:04:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 -- start start 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c1a0127380 con 0x55c1a01c26f0 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c1a00c3c30 con 0x55c1a01c1dd0 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb25e40b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c1a0123340 con 0x55c1a01ccfb0 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58408/0 (socket says 172.21.15.12:58408) 2024-04-26T22:04:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb258c00640 1 -- 172.21.15.12:0/2544113038 learned_addr learned my addr 172.21.15.12:0/2544113038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.070+0000 7fb253e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb253e00640 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 -- 172.21.15.12:0/2544113038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 msgr2=0x55c1a01cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 -- 172.21.15.12:0/2544113038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c1dd0 msgr2=0x55c1a01c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 -- 172.21.15.12:0/2544113038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a00e5940 con 0x55c1a01c26f0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb258c00640 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2480114c0 tx=0x7fb248011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0501bc1955f6030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb245c00640 1 -- 172.21.15.12:0/2544113038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248004020 con 0x55c1a01c26f0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb245c00640 1 -- 172.21.15.12:0/2544113038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2480041c0 con 0x55c1a01c26f0 2024-04-26T22:04:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.071+0000 7fb245c00640 1 -- 172.21.15.12:0/2544113038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800f820 con 0x55c1a01c26f0 2024-04-26T22:04:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 -- 172.21.15.12:0/2544113038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 msgr2=0x55c1a01cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb2480114c0 tx=0x7fb248011980 comp rx=0 tx=0).stop 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb251600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 -- 172.21.15.12:0/2544113038 shutdown_connections 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c26f0 0x55c1a01cca70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.072+0000 7fb25e40b740 1 --2- 172.21.15.12:0/2544113038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb25e40b740 1 -- 172.21.15.12:0/2544113038 >> 172.21.15.12:0/2544113038 conn(0x55c1a01be2e0 msgr2=0x55c1a01be6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb251600640 1 -- 172.21.15.12:0/2376138880 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fb24c0f1be0 con 0x7fb254108930 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb25e40b740 1 -- 172.21.15.12:0/2544113038 shutdown_connections 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb253e00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb25e40b740 1 -- 172.21.15.12:0/2544113038 wait complete. 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.073+0000 7fb25e40b740 1 -- start start 2024-04-26T22:04:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb253e00640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55c1a01bf700 tx=0x7fb240005850 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 Processor -- start 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c1a01cfbe0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c1a00aec60 con 0x55c1a01c26f0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c1a01cfd60 con 0x55c1a01ccfb0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb25e40b740 10 client.0 _ll_register_callbacks cb 0x55c1a00dae50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.074+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb253e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb259600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58424/0 (socket says 172.21.15.12:58424) 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 -- 172.21.15.12:0/1833965252 learned_addr learned my addr 172.21.15.12:0/1833965252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 -- 172.21.15.12:0/1833965252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 msgr2=0x55c1a01ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 -- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 msgr2=0x55c1a01ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 -- 172.21.15.12:0/1833965252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a00bd510 con 0x55c1a01c1dd0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb258c00640 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb248005f60 tx=0x7fb2480043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff8e9867da10a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb23a000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248025030 con 0x55c1a01c1dd0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.075+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fb248016070 con 0x55c1a01c1dd0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2480041c0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb24802f050 con 0x55c1a01c1dd0 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 10 client.15453 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c19fe39210 con 0x55c1a01c1dd0 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2480041c0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb248020030 con 0x55c1a01c1dd0 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 10 client.15453 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 20 client.15453 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.076+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a01dcce0 0x55c1a01df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.077+0000 7fb25e40b740 4 client.15453 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.077+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a0134f90 con 0x55c1a01dcce0 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.077+0000 7fb253e00640 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a01dcce0 0x55c1a01df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.078+0000 7fb253e00640 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a01dcce0 0x55c1a01df0d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7a6405b090bf1f server_cookie=8b2d24aa54d23317 in_seq=0 out_seq=0 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.078+0000 7fb23a000640 10 client.15453 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.078+0000 7fb23a000640 1 -- 172.21.15.12:0/1833965252 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1a0134f90 con 0x55c1a01dcce0 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb23a000640 10 client.15453 handle_command_reply: tid=0 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 client.15453 shutdown 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 msgr2=0x55c1a01c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb248005f60 tx=0x7fb2480043f0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 shutdown_connections 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a01dcce0 0x55c1a01df0d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1833965252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01c9f40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 >> 172.21.15.12:0/1833965252 conn(0x55c1a01be2e0 msgr2=0x55c1a01c4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb23a000640 0 client.15453 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 shutdown_connections 2024-04-26T22:04:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 1 -- 172.21.15.12:0/1833965252 wait complete. 2024-04-26T22:04:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.079+0000 7fb25e40b740 20 client.15453 trim_cache size 0 max 16384 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 Processor -- start 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 -- start start 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541afde0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541b2660 con 0x55c1a01c26f0 2024-04-26T22:04:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.096+0000 7fb25e40b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541b0450 con 0x55c1a01ccfb0 2024-04-26T22:04:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58432/0 (socket says 172.21.15.12:58432) 2024-04-26T22:04:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 -- 172.21.15.12:0/4005356233 learned_addr learned my addr 172.21.15.12:0/4005356233 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb259600640 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb253e00640 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 -- 172.21.15.12:0/4005356233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 msgr2=0x55c1a01e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 -- 172.21.15.12:0/4005356233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 msgr2=0x55c1a01bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01bd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 -- 172.21.15.12:0/4005356233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19ff05620 con 0x55c1a01c1dd0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb258c00640 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb248011e80 tx=0x7fb248004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e625b574d2c4cf62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.097+0000 7fb245200640 1 -- 172.21.15.12:0/4005356233 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248015020 con 0x55c1a01c1dd0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb245200640 1 -- 172.21.15.12:0/4005356233 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24800f5c0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb245200640 1 -- 172.21.15.12:0/4005356233 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248025040 con 0x55c1a01c1dd0 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 -- 172.21.15.12:0/4005356233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 msgr2=0x55c1a01d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb248011e80 tx=0x7fb248004380 comp rx=0 tx=0).stop 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 -- 172.21.15.12:0/4005356233 shutdown_connections 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01ccfb0 0x55c1a01e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01c26f0 0x55c1a01bd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 --2- 172.21.15.12:0/4005356233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d8aa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 -- 172.21.15.12:0/4005356233 >> 172.21.15.12:0/4005356233 conn(0x55c1a01be2e0 msgr2=0x55c1a01bcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 -- 172.21.15.12:0/4005356233 shutdown_connections 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.098+0000 7fb25e40b740 1 -- 172.21.15.12:0/4005356233 wait complete. 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 -- start start 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 Processor -- start 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 0x55c1a01d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 0x55c1a023b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541b0b20 con 0x55c1a01c1dd0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541b1f90 con 0x55c1a01ccfb0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.099+0000 7fb25e40b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2541b11f0 con 0x55c1a01c26f0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb25e40b740 10 client.0 _ll_register_callbacks cb 0x7fb240009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58442/0 (socket says 172.21.15.12:58442) 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 -- 172.21.15.12:0/1092470183 learned_addr learned my addr 172.21.15.12:0/1092470183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb259600640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 0x55c1a023b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb253e00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 0x55c1a01d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 -- 172.21.15.12:0/1092470183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 msgr2=0x55c1a01d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 0x55c1a01d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 -- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 msgr2=0x55c1a023b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 0x55c1a023b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 -- 172.21.15.12:0/1092470183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a00e1cb0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb253e00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 0x55c1a01d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb259600640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 0x55c1a023b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb258c00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb248009380 tx=0x7fb248005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5403e1fc46022f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.100+0000 7fb23a000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248015070 con 0x55c1a01c1dd0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fb2480037f0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24800f5c0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb248025070 con 0x55c1a01c1dd0 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb25e40b740 10 client.15459 fetch_fsmap learned FSMap version 21 2024-04-26T22:04:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c1a0242880 con 0x55c1a01c1dd0 2024-04-26T22:04:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.101+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24800f5c0 con 0x55c1a01c1dd0 2024-04-26T22:04:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb24803b070 con 0x55c1a01c1dd0 2024-04-26T22:04:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 10 client.15459 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:04:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 20 client.15459 populate_metadata read hostname 'smithi012' 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a0242d00 0x55c1a02450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 4 client.15459 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1a012ee00 con 0x55c1a0242d00 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.102+0000 7fb253e00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a0242d00 0x55c1a02450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.103+0000 7fb253e00640 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a0242d00 0x55c1a02450f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8129f049cc1761c7 server_cookie=af749176de24b290 in_seq=0 out_seq=0 2024-04-26T22:04:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.103+0000 7fb23a000640 10 client.15459 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb23a000640 1 -- 172.21.15.12:0/1092470183 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1a012ee00 con 0x55c1a0242d00 2024-04-26T22:04:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb23a000640 10 client.15459 handle_command_reply: tid=0 2024-04-26T22:04:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 client.15459 shutdown 2024-04-26T22:04:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 msgr2=0x55c1a01d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb248009380 tx=0x7fb248005cc0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 shutdown_connections 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55c1a0242d00 0x55c1a02450f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c1a01ccfb0 0x55c1a023b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c1a01c26f0 0x55c1a01d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 --2- 172.21.15.12:0/1092470183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1a01c1dd0 0x55c1a01d0280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb23a000640 0 client.15459 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.116+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 >> 172.21.15.12:0/1092470183 conn(0x55c1a01be2e0 msgr2=0x55c1a01bd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.117+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 shutdown_connections 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.117+0000 7fb25e40b740 1 -- 172.21.15.12:0/1092470183 wait complete. 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.117+0000 7fb25e40b740 20 client.15459 trim_cache size 0 max 16384 2024-04-26T22:04:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:04:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:04:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:04:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:04:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 msgr2=0x7fb23c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55c1a01bf700 tx=0x7fb240005850 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 msgr2=0x7fb25419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005e30 tx=0x7fb24c030f50 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 shutdown_connections 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb254108930 0x7fb25419fb10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2541032e0 0x7fb25419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 --2- 172.21.15.12:0/2376138880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb254102920 0x7fb25419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 >> 172.21.15.12:0/2376138880 conn(0x7fb2540fe3f0 msgr2=0x7fb2540fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 shutdown_connections 2024-04-26T22:04:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:04:51.118+0000 7fb25b600640 1 -- 172.21.15.12:0/2376138880 wait complete. 2024-04-26T22:04:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:51 smithi077 ceph-mon[32280]: pgmap v70: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:51.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:51 smithi012 ceph-mon[28184]: pgmap v70: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:51.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:51 smithi154 ceph-mon[32578]: pgmap v70: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 re-readers = 3392780.25 kB/sec 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 re-readers = 3383847.24 kB/sec 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 3392780.25 kB/sec 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 3392780.25 kB/sec 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 3392780.25 kB/sec 2024-04-26T22:04:53.481 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 1048576.00 kB 2024-04-26T22:04:53.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:53 smithi077 ceph-mon[32280]: pgmap v71: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:53 smithi012 ceph-mon[28184]: pgmap v71: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:53 smithi154 ceph-mon[32578]: pgmap v71: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 147 MiB/s wr, 65 op/s 2024-04-26T22:04:55.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:55 smithi077 ceph-mon[32280]: pgmap v72: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-04-26T22:04:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:55 smithi012 ceph-mon[28184]: pgmap v72: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-04-26T22:04:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:55 smithi154 ceph-mon[32578]: pgmap v72: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 171 MiB/s wr, 76 op/s 2024-04-26T22:04:57.443 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:57 smithi154 ceph-mon[32578]: pgmap v73: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:57.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:57 smithi077 ceph-mon[32280]: pgmap v73: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:57 smithi012 ceph-mon[28184]: pgmap v73: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T22:04:59.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:04:59 smithi077 ceph-mon[32280]: pgmap v74: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-04-26T22:04:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:04:59 smithi012 ceph-mon[28184]: pgmap v74: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-04-26T22:04:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:04:59 smithi154 ceph-mon[32578]: pgmap v74: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-04-26T22:04:59.761 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.761 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.761 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.761 INFO:tasks.workunit.client.0.smithi012.stdout:iozone test complete. 2024-04-26T22:04:59.768 INFO:tasks.workunit.client.0.smithi012.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T22:04:59.860 INFO:tasks.workunit.client.0.smithi012.stdout: Iozone: Performance Test of File I/O 2024-04-26T22:04:59.860 INFO:tasks.workunit.client.0.smithi012.stdout: Version $Revision: 3.494 $ 2024-04-26T22:04:59.860 INFO:tasks.workunit.client.0.smithi012.stdout: Compiled for 64 bit mode. 2024-04-26T22:04:59.860 INFO:tasks.workunit.client.0.smithi012.stdout: Build: linux-AMD64 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Run began: Fri Apr 26 22:04:59 2024 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:04:59.861 INFO:tasks.workunit.client.0.smithi012.stdout: Include close in write timing 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Include fsync in write timing 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: File size set to 10485760 kB 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Record Size 1024 kB 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Output is in kBytes/sec 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Processor cache line size set to 32 bytes. 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: File stride size set to 17 * record size. 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Throughput test with 1 process 2024-04-26T22:04:59.862 INFO:tasks.workunit.client.0.smithi012.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-26T22:05:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:01 smithi077 ceph-mon[32280]: pgmap v75: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:01 smithi012 ceph-mon[28184]: pgmap v75: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:01.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:01 smithi154 ceph-mon[32578]: pgmap v75: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:03.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:03 smithi077 ceph-mon[32280]: pgmap v76: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:03.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:03 smithi012 ceph-mon[28184]: pgmap v76: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:03 smithi154 ceph-mon[32578]: pgmap v76: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 MiB/s wr, 11 op/s 2024-04-26T22:05:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.315+0000 7f185e600640 1 -- 172.21.15.12:0/1240235719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858076e60 msgr2=0x7f1858077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.315+0000 7f185e600640 1 --2- 172.21.15.12:0/1240235719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858076e60 0x7f1858077240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1848009930 tx=0x7f184802a420 comp rx=0 tx=0).stop 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 -- 172.21.15.12:0/1240235719 shutdown_connections 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 --2- 172.21.15.12:0/1240235719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18581137c0 0x7f1858115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 --2- 172.21.15.12:0/1240235719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1858077780 0x7f1858113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 --2- 172.21.15.12:0/1240235719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858076e60 0x7f1858077240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 -- 172.21.15.12:0/1240235719 >> 172.21.15.12:0/1240235719 conn(0x7f1858100630 msgr2=0x7f1858102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 -- 172.21.15.12:0/1240235719 shutdown_connections 2024-04-26T22:05:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.316+0000 7f185e600640 1 -- 172.21.15.12:0/1240235719 wait complete. 2024-04-26T22:05:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.317+0000 7f185e600640 1 Processor -- start 2024-04-26T22:05:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 -- start start 2024-04-26T22:05:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1858076e60 0x7f1858073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18581137c0 0x7f185806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1858119290 con 0x7f1858076e60 2024-04-26T22:05:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1858119110 con 0x7f18581137c0 2024-04-26T22:05:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f185e600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1858119410 con 0x7f1858077780 2024-04-26T22:05:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.318+0000 7f1856a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:39842/0 (socket says 172.21.15.12:39842) 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 -- 172.21.15.12:0/628749013 learned_addr learned my addr 172.21.15.12:0/628749013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 -- 172.21.15.12:0/628749013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18581137c0 msgr2=0x7f185806cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18581137c0 0x7f185806cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 -- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1858076e60 msgr2=0x7f1858073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1858076e60 0x7f1858073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 -- 172.21.15.12:0/628749013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185806d830 con 0x7f1858077780 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.319+0000 7f1856a00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f184400b4a0 tx=0x7f184400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.320+0000 7f184fe00640 1 -- 172.21.15.12:0/628749013 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844014070 con 0x7f1858077780 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.320+0000 7f184fe00640 1 -- 172.21.15.12:0/628749013 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1844010ac0 con 0x7f1858077780 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.320+0000 7f184fe00640 1 -- 172.21.15.12:0/628749013 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844017c00 con 0x7f1858077780 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.321+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185806db20 con 0x7f1858077780 2024-04-26T22:05:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.321+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858071bd0 con 0x7f1858077780 2024-04-26T22:05:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.328+0000 7f18615c6740 1 Processor -- start 2024-04-26T22:05:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.328+0000 7f18615c6740 1 -- start start 2024-04-26T22:05:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe34fb0 0x5601dbe373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd8f380 con 0x5601dbe2a6f0 2024-04-26T22:05:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd2bc30 con 0x5601dbe29dd0 2024-04-26T22:05:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.329+0000 7f18615c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd8b340 con 0x5601dbe34fb0 2024-04-26T22:05:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.333+0000 7f1857e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.333+0000 7f1857e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60690/0 (socket says 172.21.15.12:60690) 2024-04-26T22:05:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.333+0000 7f1857e00640 1 -- 172.21.15.12:0/2881966834 learned_addr learned my addr 172.21.15.12:0/2881966834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.333+0000 7f1857400640 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 -- 172.21.15.12:0/2881966834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe34fb0 msgr2=0x5601dbe373a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe34fb0 0x5601dbe373a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f184fe00640 1 -- 172.21.15.12:0/628749013 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1844017070 con 0x7f1858077780 2024-04-26T22:05:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 -- 172.21.15.12:0/2881966834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 msgr2=0x5601dbe2a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 -- 172.21.15.12:0/2881966834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dbd4d940 con 0x5601dbe2a6f0 2024-04-26T22:05:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.335+0000 7f1857400640 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe34a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f1848032410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8998fd5b1018487 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.338+0000 7f1857e00640 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.338+0000 7f184cc00640 1 -- 172.21.15.12:0/2881966834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184803e030 con 0x5601dbe2a6f0 2024-04-26T22:05:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.338+0000 7f184cc00640 1 -- 172.21.15.12:0/2881966834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848004260 con 0x5601dbe2a6f0 2024-04-26T22:05:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.339+0000 7f184cc00640 1 -- 172.21.15.12:0/2881966834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848035040 con 0x5601dbe2a6f0 2024-04-26T22:05:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.343+0000 7f184fe00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 0x7f183c0a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.343+0000 7f18615c6740 1 -- 172.21.15.12:0/2881966834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 msgr2=0x5601dbe34a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.343+0000 7f18615c6740 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe34a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1848000c00 tx=0x7f1848032410 comp rx=0 tx=0).stop 2024-04-26T22:05:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.343+0000 7f184fe00640 1 -- 172.21.15.12:0/628749013 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f18440cfe30 con 0x7f1858077780 2024-04-26T22:05:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.344+0000 7f1857e00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 0x7f183c0a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 -- 172.21.15.12:0/2881966834 shutdown_connections 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe34fb0 0x5601dbe373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe34a70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 --2- 172.21.15.12:0/2881966834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 -- 172.21.15.12:0/2881966834 >> 172.21.15.12:0/2881966834 conn(0x5601dbe262e0 msgr2=0x5601dbe266c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 -- 172.21.15.12:0/2881966834 shutdown_connections 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.345+0000 7f18615c6740 1 -- 172.21.15.12:0/2881966834 wait complete. 2024-04-26T22:05:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.346+0000 7f18615c6740 1 -- start start 2024-04-26T22:05:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.347+0000 7f18615c6740 1 Processor -- start 2024-04-26T22:05:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.347+0000 7f1857e00640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 0x7f183c0a6460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1850005ee0 tx=0x7f1850005e50 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5601dbe37b80 con 0x5601dbe34fb0 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5601dbd16c60 con 0x5601dbe2a6f0 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f1856a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f1856a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:60696/0 (socket says 172.21.15.12:60696) 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f1856a00640 1 -- 172.21.15.12:0/199084577 learned_addr learned my addr 172.21.15.12:0/199084577 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f1857400640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f1857e00640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5601dbe37d00 con 0x5601dbe29dd0 2024-04-26T22:05:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.348+0000 7f18615c6740 10 client.0 _ll_register_callbacks cb 0x5601dbd42e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1856a00640 1 -- 172.21.15.12:0/199084577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 msgr2=0x5601dbe31f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1856a00640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe31f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1856a00640 1 -- 172.21.15.12:0/199084577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 msgr2=0x5601dbe329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1856a00640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe329b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1856a00640 1 -- 172.21.15.12:0/199084577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dbd25510 con 0x5601dbe2a6f0 2024-04-26T22:05:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.350+0000 7f1857e00640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.351+0000 7f1841600640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:05:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.351+0000 7f1856a00640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f184409b7d0 tx=0x7f184409c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7defe70faa97c40d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.356+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184401e070 con 0x5601dbe2a6f0 2024-04-26T22:05:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.356+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f1844099dd0 con 0x5601dbe2a6f0 2024-04-26T22:05:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.358+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1848004260 con 0x5601dbe2a6f0 2024-04-26T22:05:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.359+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f18440a4740 con 0x5601dbe2a6f0 2024-04-26T22:05:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.359+0000 7f18615c6740 10 client.25592 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.359+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601dbaa1210 con 0x5601dbe2a6f0 2024-04-26T22:05:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.361+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1848004260 con 0x5601dbe2a6f0 2024-04-26T22:05:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.362+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f18440cd3f0 con 0x5601dbe2a6f0 2024-04-26T22:05:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.362+0000 7f18615c6740 10 client.25592 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.362+0000 7f18615c6740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.362+0000 7f18615c6740 20 client.25592 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.363+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbe44d70 0x5601dbe47160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.363+0000 7f18615c6740 4 client.25592 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.363+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601dbd9cf90 con 0x5601dbe44d70 2024-04-26T22:05:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.363+0000 7f1857400640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbe44d70 0x5601dbe47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.364+0000 7f1841600640 10 client.25592 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.364+0000 7f1857400640 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbe44d70 0x5601dbe47160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0811f35988a29f server_cookie=f35e0e9f7ab12e5 in_seq=0 out_seq=0 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.365+0000 7f1841600640 1 -- 172.21.15.12:0/199084577 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601dbd9cf90 con 0x5601dbe44d70 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.365+0000 7f1841600640 10 client.25592 handle_command_reply: tid=0 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.365+0000 7f18615c6740 1 client.25592 shutdown 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.366+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 msgr2=0x5601dbe32470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.366+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f184409b7d0 tx=0x7f184409c1d0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 shutdown_connections 2024-04-26T22:05:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbe44d70 0x5601dbe47160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe2a6f0 0x5601dbe32470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 --2- 172.21.15.12:0/199084577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.367+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 >> 172.21.15.12:0/199084577 conn(0x5601dbe262e0 msgr2=0x5601dbe2c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.368+0000 7f1841600640 0 client.25592 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.370+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 shutdown_connections 2024-04-26T22:05:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.370+0000 7f18615c6740 1 -- 172.21.15.12:0/199084577 wait complete. 2024-04-26T22:05:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.371+0000 7f18615c6740 20 client.25592 trim_cache size 0 max 16384 2024-04-26T22:05:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 Processor -- start 2024-04-26T22:05:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 -- start start 2024-04-26T22:05:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 0x5601dbe327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bc8e0 con 0x5601dbe34fb0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bf1a0 con 0x5601dbe29dd0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f18615c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bcf90 con 0x5601dbe2a6f0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f1857e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.407+0000 7f1856a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 0x5601dbe327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1856a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 0x5601dbe327d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:39872/0 (socket says 172.21.15.12:39872) 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1856a00640 1 -- 172.21.15.12:0/793865423 learned_addr learned my addr 172.21.15.12:0/793865423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857400640 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 -- 172.21.15.12:0/793865423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 msgr2=0x5601dbe327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 0x5601dbe327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 -- 172.21.15.12:0/793865423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 msgr2=0x5601dbe2c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 -- 172.21.15.12:0/793865423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dbe30f00 con 0x5601dbe34fb0 2024-04-26T22:05:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.408+0000 7f1857e00640 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe252c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1850010bd0 tx=0x7f1850013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f30b94ef30a743 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.409+0000 7f1843e00640 1 -- 172.21.15.12:0/793865423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850016070 con 0x5601dbe34fb0 2024-04-26T22:05:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.409+0000 7f1843e00640 1 -- 172.21.15.12:0/793865423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1850022070 con 0x5601dbe34fb0 2024-04-26T22:05:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.409+0000 7f1843e00640 1 -- 172.21.15.12:0/793865423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850026420 con 0x5601dbe34fb0 2024-04-26T22:05:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 -- 172.21.15.12:0/793865423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 msgr2=0x5601dbe252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe252c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1850010bd0 tx=0x7f1850013000 comp rx=0 tx=0).stop 2024-04-26T22:05:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 -- 172.21.15.12:0/793865423 shutdown_connections 2024-04-26T22:05:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe34fb0 0x5601dbe252c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe2a6f0 0x5601dbe327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 --2- 172.21.15.12:0/793865423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe29dd0 0x5601dbe2c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.410+0000 7f18615c6740 1 -- 172.21.15.12:0/793865423 >> 172.21.15.12:0/793865423 conn(0x5601dbe262e0 msgr2=0x5601dbe24bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.411+0000 7f18615c6740 1 -- 172.21.15.12:0/793865423 shutdown_connections 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.411+0000 7f18615c6740 1 -- 172.21.15.12:0/793865423 wait complete. 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.411+0000 7f18615c6740 1 -- start start 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.411+0000 7f18615c6740 1 Processor -- start 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe34fb0 0x5601dbe2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bd660 con 0x5601dbe2a6f0 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bead0 con 0x5601dbe34fb0 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18581bdd30 con 0x5601dbe29dd0 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.412+0000 7f18615c6740 10 client.0 _ll_register_callbacks cb 0x7f184409b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:39884/0 (socket says 172.21.15.12:39884) 2024-04-26T22:05:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 -- 172.21.15.12:0/4133827523 learned_addr learned my addr 172.21.15.12:0/4133827523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1856a00640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 -- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe34fb0 msgr2=0x5601dbe2f840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe34fb0 0x5601dbe2f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 -- 172.21.15.12:0/4133827523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 msgr2=0x5601dbe2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.413+0000 7f1857400640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.414+0000 7f1857400640 1 -- 172.21.15.12:0/4133827523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dbb6d620 con 0x5601dbe29dd0 2024-04-26T22:05:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.414+0000 7f1856a00640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.415+0000 7f1857400640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1848005ec0 tx=0x7f18480035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daaa0254345895d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.415+0000 7f1841600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:05:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.416+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184803e040 con 0x5601dbe29dd0 2024-04-26T22:05:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.416+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f1848003890 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.417+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1850022070 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.417+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1848035040 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.417+0000 7f18615c6740 10 client.25588 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.417+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601dbd49cb0 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.417+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1850022070 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1848032500 con 0x5601dbe29dd0 2024-04-26T22:05:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 10 client.25588 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 20 client.25588 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbeab5d0 0x5601dbead9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 4 client.25588 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601dbd96e00 con 0x5601dbeab5d0 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.418+0000 7f1856a00640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbeab5d0 0x5601dbead9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.419+0000 7f1856a00640 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbeab5d0 0x5601dbead9c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba34807ab0eee0bf server_cookie=dc20e079746281e8 in_seq=0 out_seq=0 2024-04-26T22:05:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.419+0000 7f1841600640 10 client.25588 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f1841600640 1 -- 172.21.15.12:0/4133827523 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601dbd96e00 con 0x5601dbeab5d0 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f1841600640 10 client.25588 handle_command_reply: tid=0 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 client.25588 shutdown 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 msgr2=0x5601dbe2edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1848005ec0 tx=0x7f18480035f0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 shutdown_connections 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5601dbeab5d0 0x5601dbead9c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5601dbe34fb0 0x5601dbe2f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601dbe2a6f0 0x5601dbe2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 --2- 172.21.15.12:0/4133827523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601dbe29dd0 0x5601dbe2edc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 >> 172.21.15.12:0/4133827523 conn(0x5601dbe262e0 msgr2=0x5601dbe25440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f1841600640 0 client.25588 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 shutdown_connections 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.430+0000 7f18615c6740 1 -- 172.21.15.12:0/4133827523 wait complete. 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.431+0000 7f18615c6740 20 client.25588 trim_cache size 0 max 16384 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:05:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.431+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 msgr2=0x7f183c0a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.431+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 0x7f183c0a6460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1850005ee0 tx=0x7f1850005e50 comp rx=0 tx=0).stop 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.431+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 msgr2=0x7f1858074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.431+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f184400b4a0 tx=0x7f184400b960 comp rx=0 tx=0).stop 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 shutdown_connections 2024-04-26T22:05:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f183c0a3fa0 0x7f183c0a6460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18581137c0 0x7f185806cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1858077780 0x7f1858074410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 --2- 172.21.15.12:0/628749013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1858076e60 0x7f1858073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 >> 172.21.15.12:0/628749013 conn(0x7f1858100630 msgr2=0x7f1858113ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 shutdown_connections 2024-04-26T22:05:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:04.432+0000 7f185e600640 1 -- 172.21.15.12:0/628749013 wait complete. 2024-04-26T22:05:05.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:05 smithi077 ceph-mon[32280]: pgmap v77: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 29 op/s 2024-04-26T22:05:05.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:05 smithi012 ceph-mon[28184]: pgmap v77: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 29 op/s 2024-04-26T22:05:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:05 smithi154 ceph-mon[32578]: pgmap v77: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 29 op/s 2024-04-26T22:05:07.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:07 smithi077 ceph-mon[32280]: pgmap v78: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-26T22:05:07.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:07 smithi012 ceph-mon[28184]: pgmap v78: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-26T22:05:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:07 smithi154 ceph-mon[32578]: pgmap v78: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-26T22:05:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:09 smithi012 ceph-mon[28184]: pgmap v79: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 67 op/s 2024-04-26T22:05:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:09 smithi154 ceph-mon[32578]: pgmap v79: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 67 op/s 2024-04-26T22:05:09.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:09 smithi077 ceph-mon[32280]: pgmap v79: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 67 op/s 2024-04-26T22:05:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:11 smithi012 ceph-mon[28184]: pgmap v80: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:11 smithi154 ceph-mon[32578]: pgmap v80: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:11.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:11 smithi077 ceph-mon[32280]: pgmap v80: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:13 smithi012 ceph-mon[28184]: pgmap v81: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:13 smithi154 ceph-mon[32578]: pgmap v81: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:13 smithi077 ceph-mon[32280]: pgmap v81: 129 pgs: 129 active+clean; 1002 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 83 MiB/s wr, 66 op/s 2024-04-26T22:05:15.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:15 smithi012 ceph-mon[28184]: pgmap v82: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 206 MiB/s wr, 99 op/s 2024-04-26T22:05:15.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:15 smithi154 ceph-mon[32578]: pgmap v82: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 206 MiB/s wr, 99 op/s 2024-04-26T22:05:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:15 smithi077 ceph-mon[32280]: pgmap v82: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 206 MiB/s wr, 99 op/s 2024-04-26T22:05:17.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:17 smithi012 ceph-mon[28184]: pgmap v83: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 81 op/s 2024-04-26T22:05:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.637+0000 7fa2a2800640 1 -- 172.21.15.12:0/191892386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c10a550 msgr2=0x7fa29c10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.637+0000 7fa2a2800640 1 --2- 172.21.15.12:0/191892386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c10a550 0x7fa29c10ddd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ca0 tx=0x7fa29002a8e0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.637+0000 7fa2a2800640 1 -- 172.21.15.12:0/191892386 shutdown_connections 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 --2- 172.21.15.12:0/191892386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 --2- 172.21.15.12:0/191892386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c10a550 0x7fa29c10ddd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 --2- 172.21.15.12:0/191892386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c109ba0 0x7fa29c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 -- 172.21.15.12:0/191892386 >> 172.21.15.12:0/191892386 conn(0x7fa29c06d870 msgr2=0x7fa29c06dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 -- 172.21.15.12:0/191892386 shutdown_connections 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.638+0000 7fa2a2800640 1 -- 172.21.15.12:0/191892386 wait complete. 2024-04-26T22:05:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.639+0000 7fa2a2800640 1 Processor -- start 2024-04-26T22:05:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.639+0000 7fa2a2800640 1 -- start start 2024-04-26T22:05:17.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:17 smithi154 ceph-mon[32578]: pgmap v83: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 81 op/s 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.640+0000 7fa2a2800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.640+0000 7fa2a2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c1a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.640+0000 7fa2a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 0x7fa29c1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.641+0000 7fa2a2800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c11a0f0 con 0x7fa29c1a3680 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.641+0000 7fa2a2800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c119f70 con 0x7fa29c109ba0 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.641+0000 7fa2a1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.642+0000 7fa2a1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:33084/0 (socket says 172.21.15.12:33084) 2024-04-26T22:05:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.642+0000 7fa2a1400640 1 -- 172.21.15.12:0/3423626802 learned_addr learned my addr 172.21.15.12:0/3423626802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.642+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c11a270 con 0x7fa29c10e4b0 2024-04-26T22:05:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.642+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 0x7fa29c1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c1a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1e00640 1 -- 172.21.15.12:0/3423626802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 msgr2=0x7fa29c1a9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c1a9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1e00640 1 -- 172.21.15.12:0/3423626802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 msgr2=0x7fa29c1a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1e00640 1 -- 172.21.15.12:0/3423626802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c1a4280 con 0x7fa29c1a3680 2024-04-26T22:05:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.643+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c1a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.644+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 0x7fa29c1a3b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c970 tx=0x7fa29400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.645+0000 7fa29a000640 1 -- 172.21.15.12:0/3423626802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294007bb0 con 0x7fa29c1a3680 2024-04-26T22:05:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.645+0000 7fa29a000640 1 -- 172.21.15.12:0/3423626802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa294007d50 con 0x7fa29c1a3680 2024-04-26T22:05:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.645+0000 7fa29a000640 1 -- 172.21.15.12:0/3423626802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294018660 con 0x7fa29c1a3680 2024-04-26T22:05:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.648+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c10ffd0 con 0x7fa29c1a3680 2024-04-26T22:05:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.648+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c1104a0 con 0x7fa29c1a3680 2024-04-26T22:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.656+0000 7fa2a6de6740 1 Processor -- start 2024-04-26T22:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.660+0000 7fa29a000640 1 -- 172.21.15.12:0/3423626802 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa29400f450 con 0x7fa29c1a3680 2024-04-26T22:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.660+0000 7fa2a6de6740 1 -- start start 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596f3e76380 con 0x5596f3f10fc0 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596f3e12c30 con 0x5596f3f118e0 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.664+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5596f3e72340 con 0x5596f3f1c1a0 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa29a000640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 0x7fa2840a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f1e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:42544/0 (socket says 172.21.15.12:42544) 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa2a0a00640 1 -- 172.21.15.12:0/3803349274 learned_addr learned my addr 172.21.15.12:0/3803349274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa29a000640 1 -- 172.21.15.12:0/3423626802 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa2940d1050 con 0x7fa29c1a3680 2024-04-26T22:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.665+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.666+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 0x7fa2840a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.667+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.667+0000 7fa2a1400640 1 -- 172.21.15.12:0/3803349274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 msgr2=0x5596f3f1e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.667+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f1e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.667+0000 7fa2a1400640 1 -- 172.21.15.12:0/3803349274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 msgr2=0x5596f3f113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.668+0000 7fa2a1e00640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 0x7fa2840a6320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2940ce3f0 tx=0x7fa2940046c0 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.668+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.668+0000 7fa2a1400640 1 -- 172.21.15.12:0/3803349274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f3e34940 con 0x5596f3f118e0 2024-04-26T22:05:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.669+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.670+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f1bc60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa28801cc30 tx=0x7fa28801e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51fc9107f1646d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.670+0000 7fa28e600640 1 -- 172.21.15.12:0/3803349274 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288020630 con 0x5596f3f118e0 2024-04-26T22:05:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.670+0000 7fa28e600640 1 -- 172.21.15.12:0/3803349274 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa288020c80 con 0x5596f3f118e0 2024-04-26T22:05:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.670+0000 7fa28e600640 1 -- 172.21.15.12:0/3803349274 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288028cb0 con 0x5596f3f118e0 2024-04-26T22:05:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3803349274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 msgr2=0x5596f3f1bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f1bc60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa28801cc30 tx=0x7fa28801e720 comp rx=0 tx=0).stop 2024-04-26T22:05:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3803349274 shutdown_connections 2024-04-26T22:05:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f1e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f1bc60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3803349274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.672+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3803349274 >> 172.21.15.12:0/3803349274 conn(0x5596f3f0d4d0 msgr2=0x5596f3f0d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.673+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3803349274 shutdown_connections 2024-04-26T22:05:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.674+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3803349274 wait complete. 2024-04-26T22:05:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.674+0000 7fa2a6de6740 1 -- start start 2024-04-26T22:05:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.675+0000 7fa2a6de6740 1 Processor -- start 2024-04-26T22:05:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f19620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f19b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596f3f1edd0 con 0x5596f3f10fc0 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596f3dfdc60 con 0x5596f3f118e0 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.680+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5596f3f1ef50 con 0x5596f3f1c1a0 2024-04-26T22:05:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.681+0000 7fa2a6de6740 10 client.0 _ll_register_callbacks cb 0x5596f3e29e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.684+0000 7fa2a1400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f19620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.685+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.685+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f190e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36976/0 (socket says 172.21.15.12:36976) 2024-04-26T22:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.685+0000 7fa2a0a00640 1 -- 172.21.15.12:0/2720724095 learned_addr learned my addr 172.21.15.12:0/2720724095 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f19b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a1400640 1 -- 172.21.15.12:0/2720724095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 msgr2=0x5596f3f19b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f19b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a1400640 1 -- 172.21.15.12:0/2720724095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 msgr2=0x5596f3f190e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f190e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.687+0000 7fa2a1400640 1 -- 172.21.15.12:0/2720724095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f3e0c510 con 0x5596f3f118e0 2024-04-26T22:05:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.690+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f19620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa28801c3a0 tx=0x7fa288028cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab358b51e3eff9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.690+0000 7fa282a00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:05:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.694+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288038020 con 0x5596f3f118e0 2024-04-26T22:05:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.695+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa288020c80 con 0x5596f3f118e0 2024-04-26T22:05:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.695+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa288020640 con 0x5596f3f118e0 2024-04-26T22:05:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.697+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa288038020 con 0x5596f3f118e0 2024-04-26T22:05:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.698+0000 7fa2a6de6740 10 client.25604 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.698+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5596f3b88210 con 0x5596f3f118e0 2024-04-26T22:05:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.700+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa288020c80 con 0x5596f3f118e0 2024-04-26T22:05:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.702+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa288034920 con 0x5596f3f118e0 2024-04-26T22:05:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.706+0000 7fa2a6de6740 10 client.25604 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.706+0000 7fa2a6de6740 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.706+0000 7fa2a6de6740 20 client.25604 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.706+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f2bda0 0x5596f3f2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.707+0000 7fa2a6de6740 4 client.25604 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.708+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f3e83f90 con 0x5596f3f2bda0 2024-04-26T22:05:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.709+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f2bda0 0x5596f3f2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.711+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f2bda0 0x5596f3f2e190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4cbd48f30b3f8b0 server_cookie=bf7551d760cdc05f in_seq=0 out_seq=0 2024-04-26T22:05:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.711+0000 7fa282a00640 10 client.25604 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.712+0000 7fa282a00640 1 -- 172.21.15.12:0/2720724095 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596f3e83f90 con 0x5596f3f2bda0 2024-04-26T22:05:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.712+0000 7fa282a00640 10 client.25604 handle_command_reply: tid=0 2024-04-26T22:05:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.712+0000 7fa2a6de6740 1 client.25604 shutdown 2024-04-26T22:05:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.714+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 msgr2=0x5596f3f19620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.714+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f19620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa28801c3a0 tx=0x7fa288028cc0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 shutdown_connections 2024-04-26T22:05:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f2bda0 0x5596f3f2e190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f19b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f19620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2720724095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f10fc0 0x5596f3f190e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 >> 172.21.15.12:0/2720724095 conn(0x5596f3f0d4d0 msgr2=0x5596f3f13250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.715+0000 7fa282a00640 0 client.25604 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.717+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 shutdown_connections 2024-04-26T22:05:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.718+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2720724095 wait complete. 2024-04-26T22:05:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.719+0000 7fa2a6de6740 20 client.25604 trim_cache size 0 max 16384 2024-04-26T22:05:17.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:17 smithi077 ceph-mon[32280]: pgmap v83: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 206 MiB/s wr, 81 op/s 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 Processor -- start 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 -- start start 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f10fc0 0x5596f3f27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f33cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b40d0 con 0x5596f3f118e0 2024-04-26T22:05:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b6930 con 0x5596f3f10fc0 2024-04-26T22:05:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.750+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b4720 con 0x5596f3f1c1a0 2024-04-26T22:05:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.751+0000 7fa2a1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.751+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f10fc0 0x5596f3f27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.751+0000 7fa2a1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36984/0 (socket says 172.21.15.12:36984) 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.751+0000 7fa2a1400640 1 -- 172.21.15.12:0/2302500528 learned_addr learned my addr 172.21.15.12:0/2302500528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 -- 172.21.15.12:0/2302500528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 msgr2=0x5596f3f33cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f33cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 -- 172.21.15.12:0/2302500528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f10fc0 msgr2=0x5596f3f27c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f10fc0 0x5596f3f27c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 -- 172.21.15.12:0/2302500528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f3c54620 con 0x5596f3f118e0 2024-04-26T22:05:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.752+0000 7fa2a1400640 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa288034f70 tx=0x7fa28801e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b613a188ad7af18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.753+0000 7fa28dc00640 1 -- 172.21.15.12:0/2302500528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288032420 con 0x5596f3f118e0 2024-04-26T22:05:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.753+0000 7fa28dc00640 1 -- 172.21.15.12:0/2302500528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28804b070 con 0x5596f3f118e0 2024-04-26T22:05:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.754+0000 7fa28dc00640 1 -- 172.21.15.12:0/2302500528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288032420 con 0x5596f3f118e0 2024-04-26T22:05:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.754+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2302500528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 msgr2=0x5596f3f33790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.754+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa288034f70 tx=0x7fa28801e720 comp rx=0 tx=0).stop 2024-04-26T22:05:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2302500528 shutdown_connections 2024-04-26T22:05:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f1c1a0 0x5596f3f33cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f118e0 0x5596f3f33790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/2302500528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f10fc0 0x5596f3f27c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2302500528 >> 172.21.15.12:0/2302500528 conn(0x5596f3f0d4d0 msgr2=0x5596f3f14380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.755+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2302500528 shutdown_connections 2024-04-26T22:05:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.756+0000 7fa2a6de6740 1 -- 172.21.15.12:0/2302500528 wait complete. 2024-04-26T22:05:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.756+0000 7fa2a6de6740 1 -- start start 2024-04-26T22:05:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.756+0000 7fa2a6de6740 1 Processor -- start 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.757+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.757+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f246e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:42588/0 (socket says 172.21.15.12:42588) 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a6de6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f1c1a0 0x5596f3f15700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b4df0 con 0x5596f3f1c1a0 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b6260 con 0x5596f3f118e0 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a6de6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1b54c0 con 0x5596f3f10fc0 2024-04-26T22:05:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a6de6740 10 client.0 _ll_register_callbacks cb 0x7fa290009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.758+0000 7fa2a0a00640 1 -- 172.21.15.12:0/3822611512 learned_addr learned my addr 172.21.15.12:0/3822611512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f1c1a0 0x5596f3f15700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 -- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 msgr2=0x5596f3f246e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f246e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 -- 172.21.15.12:0/3822611512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f1c1a0 msgr2=0x5596f3f15700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f1c1a0 0x5596f3f15700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a0a00640 1 -- 172.21.15.12:0/3822611512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f3e30cb0 con 0x5596f3f10fc0 2024-04-26T22:05:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.759+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f246e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:05:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.760+0000 7fa2a0a00640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ca0 tx=0x7fa290041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b335b31c487393 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.760+0000 7fa282a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:05:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.760+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290051030 con 0x5596f3f10fc0 2024-04-26T22:05:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.760+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fa29004b270 con 0x5596f3f10fc0 2024-04-26T22:05:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.760+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28804b070 con 0x5596f3f10fc0 2024-04-26T22:05:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa29004b470 con 0x5596f3f10fc0 2024-04-26T22:05:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 10 client.25600 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5596f3f91950 con 0x5596f3f10fc0 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28804b070 con 0x5596f3f10fc0 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa290030d10 con 0x5596f3f10fc0 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 10 client.25600 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 20 client.25600 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f920a0 0x5596f3f944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 4 client.25600 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:05:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.761+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596f3e7de00 con 0x5596f3f920a0 2024-04-26T22:05:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.762+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f920a0 0x5596f3f944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.762+0000 7fa2a1400640 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f920a0 0x5596f3f944c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9df95ac84671eb server_cookie=420516b8301a0381 in_seq=0 out_seq=0 2024-04-26T22:05:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.762+0000 7fa282a00640 10 client.25600 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.779+0000 7fa282a00640 1 -- 172.21.15.12:0/3822611512 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596f3e7de00 con 0x5596f3f920a0 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.779+0000 7fa282a00640 10 client.25600 handle_command_reply: tid=0 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.781+0000 7fa2a6de6740 1 client.25600 shutdown 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.781+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 msgr2=0x5596f3f241a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.781+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ca0 tx=0x7fa290041040 comp rx=0 tx=0).stop 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.781+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 shutdown_connections 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.781+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5596f3f920a0 0x5596f3f944c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596f3f1c1a0 0x5596f3f15700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3f118e0 0x5596f3f246e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 --2- 172.21.15.12:0/3822611512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596f3f10fc0 0x5596f3f241a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 >> 172.21.15.12:0/3822611512 conn(0x5596f3f0d4d0 msgr2=0x5596f3f12970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa282a00640 0 client.25600 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 shutdown_connections 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 1 -- 172.21.15.12:0/3822611512 wait complete. 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.782+0000 7fa2a6de6740 20 client.25600 trim_cache size 0 max 16384 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:05:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 msgr2=0x7fa2840a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 0x7fa2840a6320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2940ce3f0 tx=0x7fa2940046c0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 msgr2=0x7fa29c1a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 0x7fa29c1a3b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c970 tx=0x7fa29400ce30 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 shutdown_connections 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fa2840a3e60 0x7fa2840a6320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa29c1a3680 0x7fa29c1a3b00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa29c10e4b0 0x7fa29c1a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 --2- 172.21.15.12:0/3423626802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa29c109ba0 0x7fa29c1a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 >> 172.21.15.12:0/3423626802 conn(0x7fa29c06d870 msgr2=0x7fa29c1174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 shutdown_connections 2024-04-26T22:05:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:17.783+0000 7fa2a2800640 1 -- 172.21.15.12:0/3423626802 wait complete. 2024-04-26T22:05:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:19 smithi012 ceph-mon[28184]: pgmap v84: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 320 MiB/s wr, 112 op/s 2024-04-26T22:05:19.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:19 smithi154 ceph-mon[32578]: pgmap v84: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 320 MiB/s wr, 112 op/s 2024-04-26T22:05:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:19 smithi077 ceph-mon[32280]: pgmap v84: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 320 MiB/s wr, 112 op/s 2024-04-26T22:05:21.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:21 smithi012 ceph-mon[28184]: pgmap v85: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:05:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:21 smithi154 ceph-mon[32578]: pgmap v85: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:05:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:21 smithi077 ceph-mon[32280]: pgmap v85: 129 pgs: 129 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:05:23.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:23 smithi012 ceph-mon[28184]: pgmap v86: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 254 MiB/s wr, 71 op/s 2024-04-26T22:05:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:23 smithi154 ceph-mon[32578]: pgmap v86: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 254 MiB/s wr, 71 op/s 2024-04-26T22:05:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:23 smithi077 ceph-mon[32280]: pgmap v86: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 254 MiB/s wr, 71 op/s 2024-04-26T22:05:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:25 smithi154 ceph-mon[32578]: pgmap v87: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 352 MiB/s wr, 97 op/s 2024-04-26T22:05:25.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:25 smithi077 ceph-mon[32280]: pgmap v87: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 352 MiB/s wr, 97 op/s 2024-04-26T22:05:25.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:25 smithi012 ceph-mon[28184]: pgmap v87: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 352 MiB/s wr, 97 op/s 2024-04-26T22:05:27.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:27 smithi154 ceph-mon[32578]: pgmap v88: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 229 MiB/s wr, 64 op/s 2024-04-26T22:05:27.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:27 smithi077 ceph-mon[32280]: pgmap v88: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 229 MiB/s wr, 64 op/s 2024-04-26T22:05:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:27 smithi012 ceph-mon[28184]: pgmap v88: 129 pgs: 129 active+clean; 5.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 229 MiB/s wr, 64 op/s 2024-04-26T22:05:29.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:29 smithi154 ceph-mon[32578]: pgmap v89: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 377 MiB/s wr, 102 op/s 2024-04-26T22:05:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:29 smithi077 ceph-mon[32280]: pgmap v89: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 377 MiB/s wr, 102 op/s 2024-04-26T22:05:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:29 smithi012 ceph-mon[28184]: pgmap v89: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 377 MiB/s wr, 102 op/s 2024-04-26T22:05:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 -- 172.21.15.12:0/1363689444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105a40 msgr2=0x7fbf10111a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 --2- 172.21.15.12:0/1363689444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105a40 0x7fbf10111a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009930 tx=0x7fbefc02a420 comp rx=0 tx=0).stop 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 -- 172.21.15.12:0/1363689444 shutdown_connections 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 --2- 172.21.15.12:0/1363689444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105a40 0x7fbf10111a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 --2- 172.21.15.12:0/1363689444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105080 0x7fbf10105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 --2- 172.21.15.12:0/1363689444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 0x7fbf10109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 -- 172.21.15.12:0/1363689444 >> 172.21.15.12:0/1363689444 conn(0x7fbf100774c0 msgr2=0x7fbf100798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 -- 172.21.15.12:0/1363689444 shutdown_connections 2024-04-26T22:05:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.957+0000 7fbf15e00640 1 -- 172.21.15.12:0/1363689444 wait complete. 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.958+0000 7fbf15e00640 1 Processor -- start 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.958+0000 7fbf15e00640 1 -- start start 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 0x7fbf1019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 0x7fbf1019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 0x7fbf1019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10114e20 con 0x7fbf10109b70 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10114ca0 con 0x7fbf10105a40 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf15e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10114fa0 con 0x7fbf10105080 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 0x7fbf1019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf0e000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 0x7fbf1019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 0x7fbf1019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47238/0 (socket says 172.21.15.12:47238) 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.959+0000 7fbf0ea00640 1 -- 172.21.15.12:0/364460904 learned_addr learned my addr 172.21.15.12:0/364460904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0f400640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 0x7fbf1019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 -- 172.21.15.12:0/364460904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 msgr2=0x7fbf1019b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 0x7fbf1019b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 -- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 msgr2=0x7fbf1019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 0x7fbf1019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 -- 172.21.15.12:0/364460904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf101a00f0 con 0x7fbf10105a40 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf0e000640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 0x7fbf1019b640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e9f0 tx=0x7fbf0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf14a00640 1 -- 172.21.15.12:0/364460904 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000ccc0 con 0x7fbf10105a40 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf14a00640 1 -- 172.21.15.12:0/364460904 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf000108e0 con 0x7fbf10105a40 2024-04-26T22:05:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101a03e0 con 0x7fbf10105a40 2024-04-26T22:05:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.960+0000 7fbf14a00640 1 -- 172.21.15.12:0/364460904 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00018880 con 0x7fbf10105a40 2024-04-26T22:05:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.961+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a7c70 con 0x7fbf10105a40 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.964+0000 7fbf14a00640 1 -- 172.21.15.12:0/364460904 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbf00010430 con 0x7fbf10105a40 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.964+0000 7fbf18c26740 1 Processor -- start 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.964+0000 7fbf18c26740 1 -- start start 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b409d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b40a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5624b4002380 con 0x5624b409cdd0 2024-04-26T22:05:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624b3f9ec30 con 0x5624b40a7fb0 2024-04-26T22:05:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf18c26740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5624b3ffe340 con 0x5624b409d6f0 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b409d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b409d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37934/0 (socket says 172.21.15.12:37934) 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf0f400640 1 -- 172.21.15.12:0/3083326844 learned_addr learned my addr 172.21.15.12:0/3083326844 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf0e000640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.965+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b40a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 -- 172.21.15.12:0/3083326844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 msgr2=0x5624b40a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b40a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 -- 172.21.15.12:0/3083326844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 msgr2=0x5624b409d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b409d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 -- 172.21.15.12:0/3083326844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b3fc0940 con 0x5624b40a7fb0 2024-04-26T22:05:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b40a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf0e000640 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40aa3a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf000ce340 tx=0x7fbf000d96f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901d5a72adc71bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf14a00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 0x7fbef40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbefbe00640 1 -- 172.21.15.12:0/3083326844 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000e4070 con 0x5624b40a7fb0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbefbe00640 1 -- 172.21.15.12:0/3083326844 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf000d1ac0 con 0x5624b40a7fb0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbefbe00640 1 -- 172.21.15.12:0/3083326844 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000df8f0 con 0x5624b40a7fb0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.966+0000 7fbf14a00640 1 -- 172.21.15.12:0/364460904 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fbf000d0fe0 con 0x7fbf10105a40 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.967+0000 7fbf0f400640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 0x7fbef40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.967+0000 7fbf18c26740 1 -- 172.21.15.12:0/3083326844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 msgr2=0x5624b40aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.967+0000 7fbf18c26740 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40aa3a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf000ce340 tx=0x7fbf000d96f0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.967+0000 7fbf0f400640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 0x7fbef40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbefc031440 tx=0x7fbefc004080 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 -- 172.21.15.12:0/3083326844 shutdown_connections 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40aa3a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b40a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 --2- 172.21.15.12:0/3083326844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b409d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 -- 172.21.15.12:0/3083326844 >> 172.21.15.12:0/3083326844 conn(0x5624b40992e0 msgr2=0x5624b40996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 -- 172.21.15.12:0/3083326844 shutdown_connections 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 -- 172.21.15.12:0/3083326844 wait complete. 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 -- start start 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.968+0000 7fbf18c26740 1 Processor -- start 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.969+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.969+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.969+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.969+0000 7fbf18c26740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5624b40aabe0 con 0x5624b409d6f0 2024-04-26T22:05:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf18c26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624b3f89c60 con 0x5624b40a7fb0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf18c26740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5624b40aad60 con 0x5624b409cdd0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47242/0 (socket says 172.21.15.12:47242) 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 -- 172.21.15.12:0/2701365051 learned_addr learned my addr 172.21.15.12:0/2701365051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf18c26740 10 client.0 _ll_register_callbacks cb 0x5624b3fb5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0e000640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0f400640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 -- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 msgr2=0x5624b40a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 -- 172.21.15.12:0/2701365051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 msgr2=0x5624b40a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.970+0000 7fbf0ea00640 1 -- 172.21.15.12:0/2701365051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b3f98510 con 0x5624b409cdd0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf08010b40 tx=0x7fbf0800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=986e3c3cf8b30888 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08004b20 con 0x5624b409cdd0 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fbf08004cc0 con 0x5624b409cdd0 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf000d1ac0 con 0x5624b409cdd0 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf08023050 con 0x5624b409cdd0 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbf18c26740 10 client.25612 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.971+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5624b3d14210 con 0x5624b409cdd0 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf000d1ac0 con 0x5624b409cdd0 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf0801a070 con 0x5624b409cdd0 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 10 client.25612 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 20 client.25612 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b40b7d70 0x5624b40ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 4 client.25612 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.972+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b400ff90 con 0x5624b40b7d70 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.973+0000 7fbf0e000640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b40b7d70 0x5624b40ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.973+0000 7fbf0e000640 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b40b7d70 0x5624b40ba1c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e7c66d0d8bed5d server_cookie=8e9ad91b75de3245 in_seq=0 out_seq=0 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.973+0000 7fbef8c00640 10 client.25612 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbef8c00640 1 -- 172.21.15.12:0/2701365051 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624b400ff90 con 0x5624b40b7d70 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbef8c00640 10 client.25612 handle_command_reply: tid=0 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 client.25612 shutdown 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 msgr2=0x5624b40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf08010b40 tx=0x7fbf0800e350 comp rx=0 tx=0).stop 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 shutdown_connections 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b40b7d70 0x5624b40ba1c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 --2- 172.21.15.12:0/2701365051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409cdd0 0x5624b40a4eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbef8c00640 0 client.25612 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.974+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 >> 172.21.15.12:0/2701365051 conn(0x5624b40992e0 msgr2=0x5624b409f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.977+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 shutdown_connections 2024-04-26T22:05:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.977+0000 7fbf18c26740 1 -- 172.21.15.12:0/2701365051 wait complete. 2024-04-26T22:05:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.977+0000 7fbf18c26740 20 client.25612 trim_cache size 0 max 16384 2024-04-26T22:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.999+0000 7fbf18c26740 1 Processor -- start 2024-04-26T22:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:30.999+0000 7fbf18c26740 1 -- start start 2024-04-26T22:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b4094170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b41c0 con 0x5624b409cdd0 2024-04-26T22:05:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b6a20 con 0x5624b40a7fb0 2024-04-26T22:05:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf18c26740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b4810 con 0x5624b409d6f0 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf0e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b4094170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf0e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b4094170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47258/0 (socket says 172.21.15.12:47258) 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf0e000640 1 -- 172.21.15.12:0/988790389 learned_addr learned my addr 172.21.15.12:0/988790389 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.000+0000 7fbf0f400640 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 -- 172.21.15.12:0/988790389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 msgr2=0x5624b4094170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b4094170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 -- 172.21.15.12:0/988790389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 msgr2=0x5624b40a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 -- 172.21.15.12:0/988790389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b4094940 con 0x5624b409cdd0 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b40b3b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf08011440 tx=0x7fbf080103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b564efc295e09cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbefb400640 1 -- 172.21.15.12:0/988790389 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802c070 con 0x5624b409cdd0 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbefb400640 1 -- 172.21.15.12:0/988790389 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf08015070 con 0x5624b409cdd0 2024-04-26T22:05:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.001+0000 7fbefb400640 1 -- 172.21.15.12:0/988790389 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011d80 con 0x5624b409cdd0 2024-04-26T22:05:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 -- 172.21.15.12:0/988790389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 msgr2=0x5624b40b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b40b3b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf08011440 tx=0x7fbf080103d0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 -- 172.21.15.12:0/988790389 shutdown_connections 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b40a7fb0 0x5624b40a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b409d6f0 0x5624b4094170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 --2- 172.21.15.12:0/988790389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409cdd0 0x5624b40b3b00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 -- 172.21.15.12:0/988790389 >> 172.21.15.12:0/988790389 conn(0x5624b40992e0 msgr2=0x5624b40a65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 -- 172.21.15.12:0/988790389 shutdown_connections 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.002+0000 7fbf18c26740 1 -- 172.21.15.12:0/988790389 wait complete. 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 -- start start 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 Processor -- start 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b40a7fb0 0x5624b40a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b4ee0 con 0x5624b409d6f0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b6350 con 0x5624b409cdd0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101b55b0 con 0x5624b40a7fb0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf18c26740 10 client.0 _ll_register_callbacks cb 0x7fbf08007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.003+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:40892/0 (socket says 172.21.15.12:40892) 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 -- 172.21.15.12:0/4048484991 learned_addr learned my addr 172.21.15.12:0/4048484991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 -- 172.21.15.12:0/4048484991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b40a7fb0 msgr2=0x5624b40a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b40a7fb0 0x5624b40a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 -- 172.21.15.12:0/4048484991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 msgr2=0x5624b40a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0e000640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 -- 172.21.15.12:0/4048484991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b3de0620 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0e000640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbf0ea00640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000f80 tx=0x7fbf0800e8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2769274d33826efb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.004+0000 7fbef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802c040 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fbf08011b00 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf08015070 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf0803b050 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 10 client.25628 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5624b3fbccb0 con 0x5624b409cdd0 2024-04-26T22:05:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf08015070 con 0x5624b409cdd0 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf0802f070 con 0x5624b409cdd0 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 10 client.25628 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 20 client.25628 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b411e930 0x5624b4120d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 4 client.25628 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.005+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624b4009e00 con 0x5624b411e930 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.006+0000 7fbf0e000640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b411e930 0x5624b4120d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.006+0000 7fbf0e000640 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b411e930 0x5624b4120d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928c73863bb99a3f server_cookie=6c0ca410339d53b3 in_seq=0 out_seq=0 2024-04-26T22:05:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.006+0000 7fbef8c00640 10 client.25628 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbef8c00640 1 -- 172.21.15.12:0/4048484991 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624b4009e00 con 0x5624b411e930 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbef8c00640 10 client.25628 handle_command_reply: tid=0 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 client.25628 shutdown 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 msgr2=0x5624b40a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000f80 tx=0x7fbf0800e8d0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 shutdown_connections 2024-04-26T22:05:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5624b411e930 0x5624b4120d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5624b40a7fb0 0x5624b40a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5624b409d6f0 0x5624b40a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 --2- 172.21.15.12:0/4048484991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624b409cdd0 0x5624b40a1c00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 >> 172.21.15.12:0/4048484991 conn(0x5624b40992e0 msgr2=0x5624b4098440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.021+0000 7fbef8c00640 0 client.25628 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.022+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 shutdown_connections 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.022+0000 7fbf18c26740 1 -- 172.21.15.12:0/4048484991 wait complete. 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.022+0000 7fbf18c26740 20 client.25628 trim_cache size 0 max 16384 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:05:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.023+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 msgr2=0x7fbef40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.023+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 0x7fbef40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbefc031440 tx=0x7fbefc004080 comp rx=0 tx=0).stop 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.023+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 msgr2=0x7fbf1019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.023+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 0x7fbf1019b640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e9f0 tx=0x7fbf0000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 shutdown_connections 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fbef40a3e40 0x7fbef40a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf10109b70 0x7fbf1019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf10105a40 0x7fbf1019b640 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 --2- 172.21.15.12:0/364460904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf10105080 0x7fbf1019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 >> 172.21.15.12:0/364460904 conn(0x7fbf100774c0 msgr2=0x7fbf10077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 shutdown_connections 2024-04-26T22:05:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:31.024+0000 7fbf15e00640 1 -- 172.21.15.12:0/364460904 wait complete. 2024-04-26T22:05:31.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:31 smithi154 ceph-mon[32578]: pgmap v90: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 71 op/s 2024-04-26T22:05:31.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:31 smithi077 ceph-mon[32280]: pgmap v90: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 71 op/s 2024-04-26T22:05:31.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:31 smithi012 ceph-mon[28184]: pgmap v90: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 263 MiB/s wr, 71 op/s 2024-04-26T22:05:33.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:33 smithi154 ceph-mon[32578]: pgmap v91: 129 pgs: 129 active+clean; 7.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 339 MiB/s wr, 94 op/s 2024-04-26T22:05:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:33 smithi077 ceph-mon[32280]: pgmap v91: 129 pgs: 129 active+clean; 7.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 339 MiB/s wr, 94 op/s 2024-04-26T22:05:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:33 smithi012 ceph-mon[28184]: pgmap v91: 129 pgs: 129 active+clean; 7.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 339 MiB/s wr, 94 op/s 2024-04-26T22:05:35.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:35 smithi154 ceph-mon[32578]: pgmap v92: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 385 MiB/s wr, 103 op/s 2024-04-26T22:05:35.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:35 smithi077 ceph-mon[32280]: pgmap v92: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 385 MiB/s wr, 103 op/s 2024-04-26T22:05:35.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:35 smithi012 ceph-mon[28184]: pgmap v92: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 385 MiB/s wr, 103 op/s 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 initial writers = 281846.19 kB/sec 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 initial writers = 281840.64 kB/sec 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 281846.19 kB/sec 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 281846.19 kB/sec 2024-04-26T22:05:37.103 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 281846.19 kB/sec 2024-04-26T22:05:37.104 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 10485760.00 kB 2024-04-26T22:05:37.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:37 smithi154 ceph-mon[32578]: pgmap v93: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 288 MiB/s wr, 77 op/s 2024-04-26T22:05:37.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:37 smithi077 ceph-mon[32280]: pgmap v93: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 288 MiB/s wr, 77 op/s 2024-04-26T22:05:37.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:37 smithi012 ceph-mon[28184]: pgmap v93: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 288 MiB/s wr, 77 op/s 2024-04-26T22:05:39.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:39 smithi154 ceph-mon[32578]: pgmap v94: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 416 MiB/s wr, 110 op/s 2024-04-26T22:05:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:39 smithi077 ceph-mon[32280]: pgmap v94: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 416 MiB/s wr, 110 op/s 2024-04-26T22:05:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:39 smithi012 ceph-mon[28184]: pgmap v94: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 416 MiB/s wr, 110 op/s 2024-04-26T22:05:41.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:41 smithi154 ceph-mon[32578]: pgmap v95: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 268 MiB/s wr, 72 op/s 2024-04-26T22:05:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:41 smithi077 ceph-mon[32280]: pgmap v95: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 268 MiB/s wr, 72 op/s 2024-04-26T22:05:41.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:41 smithi012 ceph-mon[28184]: pgmap v95: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 268 MiB/s wr, 72 op/s 2024-04-26T22:05:43.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:43 smithi154 ceph-mon[32578]: pgmap v96: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 278 MiB/s wr, 75 op/s 2024-04-26T22:05:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:43 smithi077 ceph-mon[32280]: pgmap v96: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 278 MiB/s wr, 75 op/s 2024-04-26T22:05:43.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:43 smithi012 ceph-mon[28184]: pgmap v96: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 278 MiB/s wr, 75 op/s 2024-04-26T22:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 -- 172.21.15.12:0/4138867429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 msgr2=0x7fef6410afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 --2- 172.21.15.12:0/4138867429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6410afd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fef54009900 tx=0x7fef5402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 -- 172.21.15.12:0/4138867429 shutdown_connections 2024-04-26T22:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 --2- 172.21.15.12:0/4138867429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 0x7fef641119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 --2- 172.21.15.12:0/4138867429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef64104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 --2- 172.21.15.12:0/4138867429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6410afd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 -- 172.21.15.12:0/4138867429 >> 172.21.15.12:0/4138867429 conn(0x7fef64100620 msgr2=0x7fef64102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.193+0000 7fef6a400640 1 -- 172.21.15.12:0/4138867429 shutdown_connections 2024-04-26T22:05:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.194+0000 7fef6a400640 1 -- 172.21.15.12:0/4138867429 wait complete. 2024-04-26T22:05:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.195+0000 7fef6a400640 1 Processor -- start 2024-04-26T22:05:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.195+0000 7fef6a400640 1 -- start start 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.195+0000 7fef6a400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.195+0000 7fef6a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 0x7fef6419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef6a400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef6a400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef64114de0 con 0x7fef6410abf0 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef6a400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef64114c60 con 0x7fef64104b30 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef6a400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef64114f60 con 0x7fef641054f0 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:50222/0 (socket says 172.21.15.12:50222) 2024-04-26T22:05:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 -- 172.21.15.12:0/310976990 learned_addr learned my addr 172.21.15.12:0/310976990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef62a00640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 0x7fef6419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63e00640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 -- 172.21.15.12:0/310976990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 msgr2=0x7fef6419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 0x7fef6419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 -- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 msgr2=0x7fef6419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.196+0000 7fef63400640 1 -- 172.21.15.12:0/310976990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef641a0170 con 0x7fef64104b30 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef63400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef54006010 tx=0x7fef54004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef5be00640 1 -- 172.21.15.12:0/310976990 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5403f030 con 0x7fef64104b30 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef641a0400 con 0x7fef64104b30 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef5be00640 1 -- 172.21.15.12:0/310976990 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef54003aa0 con 0x7fef64104b30 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef5be00640 1 -- 172.21.15.12:0/310976990 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54035070 con 0x7fef64104b30 2024-04-26T22:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.197+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef641a7cf0 con 0x7fef64104b30 2024-04-26T22:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.202+0000 7fef6d36a740 1 Processor -- start 2024-04-26T22:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.202+0000 7fef6d36a740 1 -- start start 2024-04-26T22:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb69414a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694173a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bb6936f380 con 0x55bb69409dd0 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb6930bc30 con 0x55bb6940a6f0 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef6d36a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bb6936b340 con 0x55bb69414fb0 2024-04-26T22:05:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef63e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef63e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59138/0 (socket says 172.21.15.12:59138) 2024-04-26T22:05:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef63e00640 1 -- 172.21.15.12:0/2483865908 learned_addr learned my addr 172.21.15.12:0/2483865908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.203+0000 7fef62a00640 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 -- 172.21.15.12:0/2483865908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 msgr2=0x55bb694173a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694173a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 -- 172.21.15.12:0/2483865908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 msgr2=0x55bb69414a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb69414a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 -- 172.21.15.12:0/2483865908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6932d940 con 0x55bb69409dd0 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef5be00640 1 -- 172.21.15.12:0/310976990 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fef54042030 con 0x7fef64104b30 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef63e00640 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637f12a1c945b57c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.204+0000 7fef58c00640 1 -- 172.21.15.12:0/2483865908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004d20 con 0x55bb69409dd0 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.205+0000 7fef58c00640 1 -- 172.21.15.12:0/2483865908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef5c004050 con 0x55bb69409dd0 2024-04-26T22:05:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.205+0000 7fef58c00640 1 -- 172.21.15.12:0/2483865908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c005280 con 0x55bb69409dd0 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.205+0000 7fef6d36a740 1 -- 172.21.15.12:0/2483865908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 msgr2=0x55bb6940a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.205+0000 7fef6d36a740 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c0118d0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.205+0000 7fef5be00640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 0x7fef480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef5be00640 1 -- 172.21.15.12:0/310976990 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fef540f10e0 con 0x7fef64104b30 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef63e00640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 0x7fef480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 -- 172.21.15.12:0/2483865908 shutdown_connections 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb69414a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 --2- 172.21.15.12:0/2483865908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940a1b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 -- 172.21.15.12:0/2483865908 >> 172.21.15.12:0/2483865908 conn(0x55bb694062e0 msgr2=0x55bb694066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 -- 172.21.15.12:0/2483865908 shutdown_connections 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.206+0000 7fef6d36a740 1 -- 172.21.15.12:0/2483865908 wait complete. 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef63e00640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 0x7fef480a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01b040 tx=0x7fef5c004910 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 -- start start 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 Processor -- start 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb694124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.207+0000 7fef6d36a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bb69417be0 con 0x55bb69409dd0 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef6d36a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb692f6c60 con 0x55bb6940a6f0 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef6d36a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bb69417d60 con 0x55bb69414fb0 2024-04-26T22:05:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59146/0 (socket says 172.21.15.12:59146) 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 -- 172.21.15.12:0/793535858 learned_addr learned my addr 172.21.15.12:0/793535858 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef6d36a740 10 client.0 _ll_register_callbacks cb 0x55bb69322e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef62a00640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb694124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63e00640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 -- 172.21.15.12:0/793535858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 msgr2=0x55bb694129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 -- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 msgr2=0x55bb694124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb694124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 -- 172.21.15.12:0/793535858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb69305510 con 0x55bb69409dd0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef62a00640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb694124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef63400640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef540bccb0 tx=0x7fef540bcbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139b8ab642f2556 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.208+0000 7fef4d600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:05:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54049040 con 0x55bb69409dd0 2024-04-26T22:05:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fef540b9390 con 0x55bb69409dd0 2024-04-26T22:05:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5c004050 con 0x55bb69409dd0 2024-04-26T22:05:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fef540bcd20 con 0x55bb69409dd0 2024-04-26T22:05:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef6d36a740 10 client.15504 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bb69081210 con 0x55bb69409dd0 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef5c004050 con 0x55bb69409dd0 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fef540c5450 con 0x55bb69409dd0 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.209+0000 7fef6d36a740 10 client.15504 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef6d36a740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef6d36a740 20 client.15504 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb6942d470 0x55bb6942f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef6d36a740 4 client.15504 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb6937cf90 con 0x55bb6942d470 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.210+0000 7fef62a00640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb6942d470 0x55bb6942f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.211+0000 7fef62a00640 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb6942d470 0x55bb6942f860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d62a6add9ec405 server_cookie=dfd188b0eb878c3c in_seq=0 out_seq=0 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.211+0000 7fef4d600640 10 client.15504 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.211+0000 7fef4d600640 1 -- 172.21.15.12:0/793535858 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb6937cf90 con 0x55bb6942d470 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.211+0000 7fef4d600640 10 client.15504 handle_command_reply: tid=0 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.211+0000 7fef6d36a740 1 client.15504 shutdown 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 msgr2=0x55bb69411f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef540bccb0 tx=0x7fef540bcbb0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 shutdown_connections 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb6942d470 0x55bb6942f860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69414fb0 0x55bb694129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb6940a6f0 0x55bb694124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef4d600640 0 client.15504 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 --2- 172.21.15.12:0/793535858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb69411f70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 >> 172.21.15.12:0/793535858 conn(0x55bb694062e0 msgr2=0x55bb6940c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 shutdown_connections 2024-04-26T22:05:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 1 -- 172.21.15.12:0/793535858 wait complete. 2024-04-26T22:05:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.212+0000 7fef6d36a740 20 client.15504 trim_cache size 0 max 16384 2024-04-26T22:05:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.232+0000 7fef6d36a740 1 Processor -- start 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.232+0000 7fef6d36a740 1 -- start start 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.232+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb6940a6f0 0x55bb6940c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb694350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef6d36a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef641ab8e0 con 0x55bb6940a6f0 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef6d36a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef641ae140 con 0x55bb69414fb0 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef6d36a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef641abf30 con 0x55bb69409dd0 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:33526/0 (socket says 172.21.15.12:33526) 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 -- 172.21.15.12:0/4035928217 learned_addr learned my addr 172.21.15.12:0/4035928217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef62a00640 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb6940a6f0 0x55bb6940c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 -- 172.21.15.12:0/4035928217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 msgr2=0x55bb694350e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb694350e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 -- 172.21.15.12:0/4035928217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb6940a6f0 msgr2=0x55bb6940c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb6940a6f0 0x55bb6940c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.233+0000 7fef63400640 1 -- 172.21.15.12:0/4035928217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6914d620 con 0x55bb69409dd0 2024-04-26T22:05:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.234+0000 7fef63400640 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fef54032a80 tx=0x7fef540bcc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a4022d10865f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.234+0000 7fef4fe00640 1 -- 172.21.15.12:0/4035928217 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54049030 con 0x55bb69409dd0 2024-04-26T22:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.234+0000 7fef4fe00640 1 -- 172.21.15.12:0/4035928217 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef540c58e0 con 0x55bb69409dd0 2024-04-26T22:05:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.234+0000 7fef4fe00640 1 -- 172.21.15.12:0/4035928217 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540ee6a0 con 0x55bb69409dd0 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 -- 172.21.15.12:0/4035928217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 msgr2=0x55bb69429330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fef54032a80 tx=0x7fef540bcc20 comp rx=0 tx=0).stop 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 -- 172.21.15.12:0/4035928217 shutdown_connections 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb694350e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb6940a6f0 0x55bb6940c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 --2- 172.21.15.12:0/4035928217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb69409dd0 0x55bb69429330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 -- 172.21.15.12:0/4035928217 >> 172.21.15.12:0/4035928217 conn(0x55bb694062e0 msgr2=0x55bb69413f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 -- 172.21.15.12:0/4035928217 shutdown_connections 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.235+0000 7fef6d36a740 1 -- 172.21.15.12:0/4035928217 wait complete. 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.236+0000 7fef6d36a740 1 -- start start 2024-04-26T22:05:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.236+0000 7fef6d36a740 1 Processor -- start 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb6940a6f0 0x55bb6940f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb6940f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef641ac600 con 0x55bb69409dd0 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef641ada70 con 0x55bb69414fb0 2024-04-26T22:05:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fef641accd0 con 0x55bb6940a6f0 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef6d36a740 10 client.0 _ll_register_callbacks cb 0x7fef54005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef63400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef63400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59170/0 (socket says 172.21.15.12:59170) 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef63400640 1 -- 172.21.15.12:0/3640085966 learned_addr learned my addr 172.21.15.12:0/3640085966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.237+0000 7fef62a00640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb6940a6f0 0x55bb6940f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63e00640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb6940f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 -- 172.21.15.12:0/3640085966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb6940a6f0 msgr2=0x55bb6940f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb6940a6f0 0x55bb6940f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 -- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 msgr2=0x55bb6940f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb6940f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 -- 172.21.15.12:0/3640085966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb69329cb0 con 0x55bb69409dd0 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef63400640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef540bcbf0 tx=0x7fef54005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2776a1c120a117 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef4d600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54049070 con 0x55bb69409dd0 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.238+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fef540035f0 con 0x55bb69409dd0 2024-04-26T22:05:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.239+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef540c58e0 con 0x55bb69409dd0 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.239+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fef540037f0 con 0x55bb69409dd0 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.239+0000 7fef6d36a740 10 client.15510 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.239+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bb69329cb0 con 0x55bb69409dd0 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.239+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef540c58e0 con 0x55bb69409dd0 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fef540c5b30 con 0x55bb69409dd0 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 10 client.15510 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 20 client.15510 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb69493fc0 0x55bb694963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 4 client.15510 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb69376e00 con 0x55bb69493fc0 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.240+0000 7fef62a00640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb69493fc0 0x55bb694963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.241+0000 7fef4d600640 10 client.15510 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.241+0000 7fef62a00640 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb69493fc0 0x55bb694963b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4259362ed4847e0 server_cookie=e3af7c971d7f0455 in_seq=0 out_seq=0 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef4d600640 1 -- 172.21.15.12:0/3640085966 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb69376e00 con 0x55bb69493fc0 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef4d600640 10 client.15510 handle_command_reply: tid=0 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 client.15510 shutdown 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 msgr2=0x55bb6940eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef540bcbf0 tx=0x7fef54005c00 comp rx=0 tx=0).stop 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 shutdown_connections 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55bb69493fc0 0x55bb694963b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb69414fb0 0x55bb6940f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bb6940a6f0 0x55bb6940f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 --2- 172.21.15.12:0/3640085966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb69409dd0 0x55bb6940eec0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 >> 172.21.15.12:0/3640085966 conn(0x55bb694062e0 msgr2=0x55bb69404bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef4d600640 0 client.15510 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.254+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 shutdown_connections 2024-04-26T22:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.255+0000 7fef6d36a740 1 -- 172.21.15.12:0/3640085966 wait complete. 2024-04-26T22:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.255+0000 7fef6d36a740 20 client.15510 trim_cache size 0 max 16384 2024-04-26T22:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 msgr2=0x7fef480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 0x7fef480a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef5c01b040 tx=0x7fef5c004910 comp rx=0 tx=0).stop 2024-04-26T22:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 msgr2=0x7fef6419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef54006010 tx=0x7fef54004270 comp rx=0 tx=0).stop 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 shutdown_connections 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fef480a3e40 0x7fef480a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6410abf0 0x7fef6419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef641054f0 0x7fef6419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 --2- 172.21.15.12:0/310976990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef64104b30 0x7fef6419b180 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef54006010 tx=0x7fef54004270 comp rx=0 tx=0).stop 2024-04-26T22:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.257+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 >> 172.21.15.12:0/310976990 conn(0x7fef64100620 msgr2=0x7fef64101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.258+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 shutdown_connections 2024-04-26T22:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:44.258+0000 7fef6a400640 1 -- 172.21.15.12:0/310976990 wait complete. 2024-04-26T22:05:45.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:45 smithi154 ceph-mon[32578]: pgmap v97: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 55 op/s 2024-04-26T22:05:45.731 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:45 smithi077 ceph-mon[32280]: pgmap v97: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 55 op/s 2024-04-26T22:05:45.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:45 smithi012 ceph-mon[28184]: pgmap v97: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 55 op/s 2024-04-26T22:05:47.634 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:47 smithi012 ceph-mon[28184]: pgmap v98: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 38 op/s 2024-04-26T22:05:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:47 smithi154 ceph-mon[32578]: pgmap v98: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 38 op/s 2024-04-26T22:05:47.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:47 smithi077 ceph-mon[32280]: pgmap v98: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 38 op/s 2024-04-26T22:05:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:49 smithi154 ceph-mon[32578]: pgmap v99: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 262 MiB/s wr, 71 op/s 2024-04-26T22:05:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:49 smithi077 ceph-mon[32280]: pgmap v99: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 262 MiB/s wr, 71 op/s 2024-04-26T22:05:49.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:49 smithi012 ceph-mon[28184]: pgmap v99: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 262 MiB/s wr, 71 op/s 2024-04-26T22:05:50.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:50 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:50.831 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:50 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:50.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:50 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:51 smithi077 ceph-mon[32280]: pgmap v100: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 134 MiB/s wr, 38 op/s 2024-04-26T22:05:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:51 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:51 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:51 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:05:51.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:51 smithi012 ceph-mon[28184]: pgmap v100: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 134 MiB/s wr, 38 op/s 2024-04-26T22:05:51.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:51 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:51.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:51 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:51 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:05:51.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:51 smithi154 ceph-mon[32578]: pgmap v100: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 134 MiB/s wr, 38 op/s 2024-04-26T22:05:51.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:51 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:51.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:51 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:51.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:51 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:05:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:53 smithi077 ceph-mon[32280]: pgmap v101: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 203 MiB/s wr, 58 op/s 2024-04-26T22:05:53.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:53 smithi012 ceph-mon[28184]: pgmap v101: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 203 MiB/s wr, 58 op/s 2024-04-26T22:05:53.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:53 smithi154 ceph-mon[32578]: pgmap v101: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 203 MiB/s wr, 58 op/s 2024-04-26T22:05:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:55 smithi077 ceph-mon[32280]: pgmap v102: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 67 op/s 2024-04-26T22:05:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:55 smithi012 ceph-mon[28184]: pgmap v102: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 67 op/s 2024-04-26T22:05:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:55 smithi154 ceph-mon[32578]: pgmap v102: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 244 MiB/s wr, 67 op/s 2024-04-26T22:05:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.456+0000 7f9c0d800640 1 -- 172.21.15.12:0/2987542971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c08076e60 msgr2=0x7f9c08077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.456+0000 7f9c0d800640 1 --2- 172.21.15.12:0/2987542971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c08076e60 0x7f9c08077240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009cd0 tx=0x7f9bf002a8f0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 -- 172.21.15.12:0/2987542971 shutdown_connections 2024-04-26T22:05:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 --2- 172.21.15.12:0/2987542971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c081135f0 0x7f9c081159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 --2- 172.21.15.12:0/2987542971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c081130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 --2- 172.21.15.12:0/2987542971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c08076e60 0x7f9c08077240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 -- 172.21.15.12:0/2987542971 >> 172.21.15.12:0/2987542971 conn(0x7f9c08100400 msgr2=0x7f9c08102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 -- 172.21.15.12:0/2987542971 shutdown_connections 2024-04-26T22:05:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.457+0000 7f9c0d800640 1 -- 172.21.15.12:0/2987542971 wait complete. 2024-04-26T22:05:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.461+0000 7f9c0d800640 1 Processor -- start 2024-04-26T22:05:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.464+0000 7f9c0d800640 1 -- start start 2024-04-26T22:05:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.464+0000 7f9c0d800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 0x7f9c0819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.465+0000 7f9c0d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.468+0000 7f9c0d800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 0x7f9c081a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.468+0000 7f9c07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.468+0000 7f9c07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58770/0 (socket says 172.21.15.12:58770) 2024-04-26T22:05:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c07400640 1 -- 172.21.15.12:0/1968399709 learned_addr learned my addr 172.21.15.12:0/1968399709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 0x7f9c0819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08119040 con 0x7f9c08077780 2024-04-26T22:05:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08118ec0 con 0x7f9c08076e60 2024-04-26T22:05:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081191c0 con 0x7f9c081135f0 2024-04-26T22:05:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.469+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 0x7f9c081a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.470+0000 7f9c07e00640 1 -- 172.21.15.12:0/1968399709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 msgr2=0x7f9c081a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.470+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 0x7f9c081a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.470+0000 7f9c07e00640 1 -- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 msgr2=0x7f9c0819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.470+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.471+0000 7f9c07e00640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c081a4520 con 0x7f9c08076e60 2024-04-26T22:05:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.471+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 0x7f9c081a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:05:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.471+0000 7f9c07400640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:05:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.471+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 0x7f9c0819f580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009ca0 tx=0x7f9bf0030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.472+0000 7f9c04c00640 1 -- 172.21.15.12:0/1968399709 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0004040 con 0x7f9c08076e60 2024-04-26T22:05:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.472+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081a47b0 con 0x7f9c08076e60 2024-04-26T22:05:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.472+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081ac0a0 con 0x7f9c08076e60 2024-04-26T22:05:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.479+0000 7f9c11c18740 1 Processor -- start 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.479+0000 7f9c04c00640 1 -- 172.21.15.12:0/1968399709 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf0040070 con 0x7f9c08076e60 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.479+0000 7f9c04c00640 1 -- 172.21.15.12:0/1968399709 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf003b790 con 0x7f9c08076e60 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.481+0000 7f9c11c18740 1 -- start start 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.481+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3c941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.481+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3cafc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 0x562de3cb25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c11c18740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562de3bf9380 con 0x562de3c946e0 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c11c18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562de3b95c30 con 0x562de3cb01b0 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c11c18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562de3bf5340 con 0x562de3c93dc0 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c07400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 0x562de3cb25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c07400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 0x562de3cb25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48778/0 (socket says 172.21.15.12:48778) 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c07400640 1 -- 172.21.15.12:0/265777249 learned_addr learned my addr 172.21.15.12:0/265777249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.482+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3c941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 -- 172.21.15.12:0/265777249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 msgr2=0x562de3cb25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 0x562de3cb25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 -- 172.21.15.12:0/265777249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 msgr2=0x562de3cafc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3cafc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 -- 172.21.15.12:0/265777249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de3bb7940 con 0x562de3c93dc0 2024-04-26T22:05:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.486+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3c941a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800ae70 tx=0x7f9bf800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c182af78cf94b950 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.487+0000 7f9c04c00640 1 -- 172.21.15.12:0/1968399709 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9bf00041e0 con 0x7f9c08076e60 2024-04-26T22:05:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.487+0000 7f9bfd600640 1 -- 172.21.15.12:0/265777249 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8005220 con 0x562de3c93dc0 2024-04-26T22:05:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.487+0000 7f9bfd600640 1 -- 172.21.15.12:0/265777249 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf80047a0 con 0x562de3c93dc0 2024-04-26T22:05:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.487+0000 7f9bfd600640 1 -- 172.21.15.12:0/265777249 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8010a80 con 0x562de3c93dc0 2024-04-26T22:05:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 -- 172.21.15.12:0/265777249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 msgr2=0x562de3c941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3c941a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800ae70 tx=0x7f9bf800f790 comp rx=0 tx=0).stop 2024-04-26T22:05:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 -- 172.21.15.12:0/265777249 shutdown_connections 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cb01b0 0x562de3cb25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3cafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 --2- 172.21.15.12:0/265777249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3c941a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 -- 172.21.15.12:0/265777249 >> 172.21.15.12:0/265777249 conn(0x562de3c904d0 msgr2=0x562de3c908b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.488+0000 7f9c11c18740 1 -- 172.21.15.12:0/265777249 shutdown_connections 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.489+0000 7f9c11c18740 1 -- 172.21.15.12:0/265777249 wait complete. 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.489+0000 7f9c11c18740 1 -- start start 2024-04-26T22:05:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.490+0000 7f9c11c18740 1 Processor -- start 2024-04-26T22:05:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.491+0000 7f9c04c00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 0x7f9be80a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.491+0000 7f9c04c00640 1 -- 172.21.15.12:0/1968399709 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f9bf00f1b80 con 0x7f9c08076e60 2024-04-26T22:05:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.492+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 0x7f9be80a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.492+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.492+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48788/0 (socket says 172.21.15.12:48788) 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c07e00640 1 -- 172.21.15.12:0/1135320365 learned_addr learned my addr 172.21.15.12:0/1135320365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 0x562de3c9c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562de3c96ea0 con 0x562de3c946e0 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562de3b80c60 con 0x562de3c93dc0 2024-04-26T22:05:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562de3c97020 con 0x562de3cb01b0 2024-04-26T22:05:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c11c18740 10 client.0 _ll_register_callbacks cb 0x562de3bace50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c07400640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 0x7f9be80a6230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000c00 tx=0x7f9bf8004f40 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.493+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 0x562de3c9c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.494+0000 7f9c07e00640 1 -- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 msgr2=0x562de3c9c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.494+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 0x562de3c9c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.494+0000 7f9c07e00640 1 -- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 msgr2=0x562de3ca1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.494+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.494+0000 7f9c07e00640 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de3b8f510 con 0x562de3c93dc0 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.495+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 0x562de3c9c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.495+0000 7f9c07400640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.495+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00ba1f0 tx=0x7f9bf00bd180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efcfa39112bc4309 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.495+0000 7f9bf6000640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:05:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.496+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00046d0 con 0x562de3c93dc0 2024-04-26T22:05:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.496+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f9bf00ef140 con 0x562de3c93dc0 2024-04-26T22:05:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.496+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf80047a0 con 0x562de3c93dc0 2024-04-26T22:05:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.499+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9bf00c1bd0 con 0x562de3c93dc0 2024-04-26T22:05:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.499+0000 7f9c11c18740 10 client.25649 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.499+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562de390b210 con 0x562de3c93dc0 2024-04-26T22:05:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.499+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf80047a0 con 0x562de3c93dc0 2024-04-26T22:05:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.499+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9bf00c6000 con 0x562de3c93dc0 2024-04-26T22:05:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 10 client.25649 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 20 client.25649 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cac5c0 0x562de3caea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 4 client.25649 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.500+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de3c06f90 con 0x562de3cac5c0 2024-04-26T22:05:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.502+0000 7f9c07400640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cac5c0 0x562de3caea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.505+0000 7f9c07400640 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cac5c0 0x562de3caea40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a178cdf97e5b52b server_cookie=d7639c723bdd3e06 in_seq=0 out_seq=0 2024-04-26T22:05:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.505+0000 7f9bf6000640 10 client.25649 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.505+0000 7f9bf6000640 1 -- 172.21.15.12:0/1135320365 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562de3c06f90 con 0x562de3cac5c0 2024-04-26T22:05:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.505+0000 7f9bf6000640 10 client.25649 handle_command_reply: tid=0 2024-04-26T22:05:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 client.25649 shutdown 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 msgr2=0x562de3ca1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00ba1f0 tx=0x7f9bf00bd180 comp rx=0 tx=0).stop 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 shutdown_connections 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cac5c0 0x562de3caea40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cb01b0 0x562de3c9c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9bf6000640 0 client.25649 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1135320365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c93dc0 0x562de3ca1510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.506+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 >> 172.21.15.12:0/1135320365 conn(0x562de3c904d0 msgr2=0x562de3c96270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.507+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 shutdown_connections 2024-04-26T22:05:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.507+0000 7f9c11c18740 1 -- 172.21.15.12:0/1135320365 wait complete. 2024-04-26T22:05:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.507+0000 7f9c11c18740 20 client.25649 trim_cache size 0 max 16384 2024-04-26T22:05:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 Processor -- start 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 -- start start 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 0x562de3c9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 0x562de3cd9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9be400df20 con 0x562de3c93dc0 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9be40107a0 con 0x562de3c946e0 2024-04-26T22:05:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.528+0000 7f9c11c18740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9be400e590 con 0x562de3cac5c0 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48804/0 (socket says 172.21.15.12:48804) 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 -- 172.21.15.12:0/4082732500 learned_addr learned my addr 172.21.15.12:0/4082732500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 0x562de3c9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 0x562de3cd9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 -- 172.21.15.12:0/4082732500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 msgr2=0x562de3cd9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 0x562de3cd9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.529+0000 7f9c07400640 1 -- 172.21.15.12:0/4082732500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 msgr2=0x562de3c9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.530+0000 7f9c07400640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 0x562de3c9b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.530+0000 7f9c07400640 1 -- 172.21.15.12:0/4082732500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de39d7620 con 0x562de3c946e0 2024-04-26T22:05:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.530+0000 7f9c07400640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c000119f0 tx=0x7f9c00005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78daae0dd6493e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.531+0000 7f9c07e00640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 0x562de3c9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:05:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.530+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 0x562de3cd9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:05:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.531+0000 7f9bfcc00640 1 -- 172.21.15.12:0/4082732500 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004220 con 0x562de3c946e0 2024-04-26T22:05:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.531+0000 7f9bfcc00640 1 -- 172.21.15.12:0/4082732500 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00024030 con 0x562de3c946e0 2024-04-26T22:05:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.531+0000 7f9bfcc00640 1 -- 172.21.15.12:0/4082732500 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000055b0 con 0x562de3c946e0 2024-04-26T22:05:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.533+0000 7f9c11c18740 1 -- 172.21.15.12:0/4082732500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 msgr2=0x562de3caf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.533+0000 7f9c11c18740 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c000119f0 tx=0x7f9c00005f90 comp rx=0 tx=0).stop 2024-04-26T22:05:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.535+0000 7f9c11c18740 1 -- 172.21.15.12:0/4082732500 shutdown_connections 2024-04-26T22:05:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.535+0000 7f9c11c18740 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3cac5c0 0x562de3cd9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.535+0000 7f9c11c18740 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3c946e0 0x562de3caf9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.535+0000 7f9c11c18740 1 --2- 172.21.15.12:0/4082732500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c93dc0 0x562de3c9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.535+0000 7f9c11c18740 1 -- 172.21.15.12:0/4082732500 >> 172.21.15.12:0/4082732500 conn(0x562de3c904d0 msgr2=0x562de3caf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.540+0000 7f9c11c18740 1 -- 172.21.15.12:0/4082732500 shutdown_connections 2024-04-26T22:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.543+0000 7f9c11c18740 1 -- 172.21.15.12:0/4082732500 wait complete. 2024-04-26T22:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.544+0000 7f9c11c18740 1 -- start start 2024-04-26T22:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.544+0000 7f9c11c18740 1 Processor -- start 2024-04-26T22:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.545+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3ca57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.545+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.545+0000 7f9c11c18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.545+0000 7f9c11c18740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9be400ec60 con 0x562de3c946e0 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.545+0000 7f9c11c18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9be40100d0 con 0x562de3cac5c0 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c0ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c0ce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48820/0 (socket says 172.21.15.12:48820) 2024-04-26T22:05:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c0ce00640 1 -- 172.21.15.12:0/1738165190 learned_addr learned my addr 172.21.15.12:0/1738165190 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:05:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9be400f330 con 0x562de3c93dc0 2024-04-26T22:05:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c11c18740 10 client.0 _ll_register_callbacks cb 0x7f9bf0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.546+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3ca57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 -- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 msgr2=0x562de3ca6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 -- 172.21.15.12:0/1738165190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 msgr2=0x562de3ca5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 -- 172.21.15.12:0/1738165190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de3bb3cb0 con 0x562de3c93dc0 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c07e00640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3ca57f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00bd7b0 tx=0x7f9bf00bd9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d14f8e8a3794c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9bf6000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:05:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.547+0000 7f9c0ce00640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.548+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00bcec0 con 0x562de3c93dc0 2024-04-26T22:05:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.548+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f9bf00beb50 con 0x562de3c93dc0 2024-04-26T22:05:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.549+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c00024030 con 0x562de3c93dc0 2024-04-26T22:05:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.549+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9bf00bdc90 con 0x562de3c93dc0 2024-04-26T22:05:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.550+0000 7f9c11c18740 10 client.25651 fetch_fsmap learned FSMap version 21 2024-04-26T22:05:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.550+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562de3d376c0 con 0x562de3c93dc0 2024-04-26T22:05:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.550+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00024030 con 0x562de3c93dc0 2024-04-26T22:05:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9bf00bc7b0 con 0x562de3c93dc0 2024-04-26T22:05:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 10 client.25651 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:05:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 20 client.25651 populate_metadata read hostname 'smithi012' 2024-04-26T22:05:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cb01b0 0x562de3d39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 4 client.25651 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:05:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562de3c00e00 con 0x562de3cb01b0 2024-04-26T22:05:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.551+0000 7f9c07400640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cb01b0 0x562de3d39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.555+0000 7f9bf6000640 10 client.25651 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.558+0000 7f9c07400640 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cb01b0 0x562de3d39c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76d905a568d7720 server_cookie=790ef538a5d0d1e6 in_seq=0 out_seq=0 2024-04-26T22:05:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.574+0000 7f9bf6000640 1 -- 172.21.15.12:0/1738165190 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562de3c00e00 con 0x562de3cb01b0 2024-04-26T22:05:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.574+0000 7f9bf6000640 10 client.25651 handle_command_reply: tid=0 2024-04-26T22:05:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.574+0000 7f9c11c18740 1 client.25651 shutdown 2024-04-26T22:05:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.574+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 msgr2=0x562de3ca57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.574+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3ca57f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00bd7b0 tx=0x7f9bf00bd9a0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 shutdown_connections 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x562de3cb01b0 0x562de3d39c90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562de3cac5c0 0x562de3ca6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562de3c946e0 0x562de3ca5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 --2- 172.21.15.12:0/1738165190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562de3c93dc0 0x562de3ca57f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 >> 172.21.15.12:0/1738165190 conn(0x562de3c904d0 msgr2=0x562de3c8eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9bf6000640 0 client.25651 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:05:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 shutdown_connections 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 1 -- 172.21.15.12:0/1738165190 wait complete. 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.575+0000 7f9c11c18740 20 client.25651 trim_cache size 0 max 16384 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:05:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.576+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 msgr2=0x7f9be80a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.576+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 0x7f9be80a6230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000c00 tx=0x7f9bf8004f40 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.576+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 msgr2=0x7f9c0819f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.576+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 0x7f9c0819f580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009ca0 tx=0x7f9bf0030da0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 shutdown_connections 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f9be80a3d70 0x7f9be80a6230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c081135f0 0x7f9c081a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c08077780 0x7f9c0819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 --2- 172.21.15.12:0/1968399709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c08076e60 0x7f9c0819f580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 >> 172.21.15.12:0/1968399709 conn(0x7f9c08100400 msgr2=0x7f9c08115140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 shutdown_connections 2024-04-26T22:05:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:05:57.577+0000 7f9c0d800640 1 -- 172.21.15.12:0/1968399709 wait complete. 2024-04-26T22:05:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:57 smithi077 ceph-mon[32280]: pgmap v103: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 65 op/s 2024-04-26T22:05:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:57 smithi012 ceph-mon[28184]: pgmap v103: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 65 op/s 2024-04-26T22:05:57.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:57 smithi154 ceph-mon[32578]: pgmap v103: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 235 MiB/s wr, 65 op/s 2024-04-26T22:05:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:05:59 smithi077 ceph-mon[32280]: pgmap v104: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T22:05:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:05:59 smithi012 ceph-mon[28184]: pgmap v104: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T22:05:59.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:05:59 smithi154 ceph-mon[32578]: pgmap v104: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 353 MiB/s wr, 96 op/s 2024-04-26T22:06:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:01 smithi012 ceph-mon[28184]: pgmap v105: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:06:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:01 smithi154 ceph-mon[32578]: pgmap v105: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:06:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:01 smithi077 ceph-mon[32280]: pgmap v105: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 236 MiB/s wr, 63 op/s 2024-04-26T22:06:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:03 smithi012 ceph-mon[28184]: pgmap v106: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 313 MiB/s wr, 87 op/s 2024-04-26T22:06:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:03 smithi154 ceph-mon[32578]: pgmap v106: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 313 MiB/s wr, 87 op/s 2024-04-26T22:06:03.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:03 smithi077 ceph-mon[32280]: pgmap v106: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 313 MiB/s wr, 87 op/s 2024-04-26T22:06:05.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:05 smithi012 ceph-mon[28184]: pgmap v107: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 289 MiB/s wr, 93 op/s 2024-04-26T22:06:05.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:05 smithi154 ceph-mon[32578]: pgmap v107: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 289 MiB/s wr, 93 op/s 2024-04-26T22:06:05.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:05 smithi077 ceph-mon[32280]: pgmap v107: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 289 MiB/s wr, 93 op/s 2024-04-26T22:06:07.826 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:07 smithi012 ceph-mon[28184]: pgmap v108: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 238 MiB/s wr, 80 op/s 2024-04-26T22:06:07.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:07 smithi154 ceph-mon[32578]: pgmap v108: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 238 MiB/s wr, 80 op/s 2024-04-26T22:06:07.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:07 smithi077 ceph-mon[32280]: pgmap v108: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 238 MiB/s wr, 80 op/s 2024-04-26T22:06:09.783 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:09 smithi077 ceph-mon[32280]: pgmap v109: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 377 MiB/s wr, 116 op/s 2024-04-26T22:06:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:09 smithi012 ceph-mon[28184]: pgmap v109: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 377 MiB/s wr, 116 op/s 2024-04-26T22:06:09.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:09 smithi154 ceph-mon[32578]: pgmap v109: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 377 MiB/s wr, 116 op/s 2024-04-26T22:06:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.760+0000 7fd6ae600640 1 -- 172.21.15.12:0/2981486950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8103110 msgr2=0x7fd6a810f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.760+0000 7fd6ae600640 1 --2- 172.21.15.12:0/2981486950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8103110 0x7fd6a810f620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009ca0 tx=0x7fd6a002a8f0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.762+0000 7fd6ae600640 1 -- 172.21.15.12:0/2981486950 shutdown_connections 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.762+0000 7fd6ae600640 1 --2- 172.21.15.12:0/2981486950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8103110 0x7fd6a810f620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.762+0000 7fd6ae600640 1 --2- 172.21.15.12:0/2981486950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8102750 0x7fd6a8102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.762+0000 7fd6ae600640 1 --2- 172.21.15.12:0/2981486950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.762+0000 7fd6ae600640 1 -- 172.21.15.12:0/2981486950 >> 172.21.15.12:0/2981486950 conn(0x7fd6a80fe280 msgr2=0x7fd6a81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.764+0000 7fd6ae600640 1 -- 172.21.15.12:0/2981486950 shutdown_connections 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.764+0000 7fd6ae600640 1 -- 172.21.15.12:0/2981486950 wait complete. 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.766+0000 7fd6ae600640 1 Processor -- start 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.766+0000 7fd6ae600640 1 -- start start 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.767+0000 7fd6ae600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 0x7fd6a810cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.768+0000 7fd6ae600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8103110 0x7fd6a810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.768+0000 7fd6ad200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 0x7fd6a810cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.769+0000 7fd6ad200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 0x7fd6a810cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:39608/0 (socket says 172.21.15.12:39608) 2024-04-26T22:06:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.769+0000 7fd6ad200640 1 -- 172.21.15.12:0/1590052173 learned_addr learned my addr 172.21.15.12:0/1590052173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.769+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8076a60 con 0x7fd6a8103110 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a80768e0 con 0x7fd6a8102750 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8073620 con 0x7fd6a8108760 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.768+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8103110 0x7fd6a810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 -- 172.21.15.12:0/1590052173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 msgr2=0x7fd6a810cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 0x7fd6a810cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 -- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8103110 msgr2=0x7fd6a810cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8103110 0x7fd6a810cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.771+0000 7fd6adc00640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8073820 con 0x7fd6a8108760 2024-04-26T22:06:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.772+0000 7fd6adc00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a810d530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0006010 tx=0x7fd6a002f3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.773+0000 7fd6a5600640 1 -- 172.21.15.12:0/1590052173 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a002fa50 con 0x7fd6a8108760 2024-04-26T22:06:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.773+0000 7fd6a5600640 1 -- 172.21.15.12:0/1590052173 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a002d070 con 0x7fd6a8108760 2024-04-26T22:06:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.773+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a8073ab0 con 0x7fd6a8108760 2024-04-26T22:06:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.774+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a8074000 con 0x7fd6a8108760 2024-04-26T22:06:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.775+0000 7fd6a5600640 1 -- 172.21.15.12:0/1590052173 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00466e0 con 0x7fd6a8108760 2024-04-26T22:06:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.784+0000 7fd6a5600640 1 -- 172.21.15.12:0/1590052173 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd6a003b400 con 0x7fd6a8108760 2024-04-26T22:06:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.786+0000 7fd6b2a53740 1 Processor -- start 2024-04-26T22:06:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.787+0000 7fd6b2a53740 1 -- start start 2024-04-26T22:06:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.788+0000 7fd6a5600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 0x7fd6900a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.788+0000 7fd6a5600640 1 -- 172.21.15.12:0/1590052173 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd6a00f1dd0 con 0x7fd6a8108760 2024-04-26T22:06:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.788+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 0x7fd6900a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f376fe0 0x55ec9f3773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f3845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f2dc380 con 0x55ec9f376fe0 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f278c30 con 0x55ec9f3821c0 2024-04-26T22:06:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.789+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f2d8340 con 0x55ec9f377900 2024-04-26T22:06:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:37496/0 (socket says 172.21.15.12:37496) 2024-04-26T22:06:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 -- 172.21.15.12:0/3602502781 learned_addr learned my addr 172.21.15.12:0/3602502781 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6adc00640 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f3845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 -- 172.21.15.12:0/3602502781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 msgr2=0x55ec9f3845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f3845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 -- 172.21.15.12:0/3602502781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f376fe0 msgr2=0x55ec9f3773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f376fe0 0x55ec9f3773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.790+0000 7fd6a7e00640 1 -- 172.21.15.12:0/3602502781 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9f29a940 con 0x55ec9f377900 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.791+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd694013830 tx=0x7fd694013cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a6a06efa4583c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.791+0000 7fd69a000640 1 -- 172.21.15.12:0/3602502781 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69400f9a0 con 0x55ec9f377900 2024-04-26T22:06:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.791+0000 7fd69a000640 1 -- 172.21.15.12:0/3602502781 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd694010430 con 0x55ec9f377900 2024-04-26T22:06:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.791+0000 7fd69a000640 1 -- 172.21.15.12:0/3602502781 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69401f620 con 0x55ec9f377900 2024-04-26T22:06:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.792+0000 7fd6b2a53740 1 -- 172.21.15.12:0/3602502781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 msgr2=0x55ec9f381c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.792+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd694013830 tx=0x7fd694013cf0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 -- 172.21.15.12:0/3602502781 shutdown_connections 2024-04-26T22:06:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f3845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f381c80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/3602502781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f376fe0 0x55ec9f3773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 -- 172.21.15.12:0/3602502781 >> 172.21.15.12:0/3602502781 conn(0x55ec9f3734f0 msgr2=0x55ec9f3738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 0x7fd6900a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd694013830 tx=0x7fd694013cf0 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 -- 172.21.15.12:0/3602502781 shutdown_connections 2024-04-26T22:06:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.793+0000 7fd6b2a53740 1 -- 172.21.15.12:0/3602502781 wait complete. 2024-04-26T22:06:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.794+0000 7fd6b2a53740 1 -- start start 2024-04-26T22:06:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.794+0000 7fd6b2a53740 1 Processor -- start 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f384df0 con 0x55ec9f377900 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f263c60 con 0x55ec9f3821c0 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ec9f384f70 con 0x55ec9f376fe0 2024-04-26T22:06:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6b2a53740 10 client.0 _ll_register_callbacks cb 0x55ec9f28fe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6ad200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6ad200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:37508/0 (socket says 172.21.15.12:37508) 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.795+0000 7fd6ad200640 1 -- 172.21.15.12:0/4034587189 learned_addr learned my addr 172.21.15.12:0/4034587189 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6adc00640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 -- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 msgr2=0x55ec9f37fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 -- 172.21.15.12:0/4034587189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 msgr2=0x55ec9f37f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 -- 172.21.15.12:0/4034587189 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9f272510 con 0x55ec9f376fe0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d1e0 tx=0x7fd69c00d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b5cee8a56f71a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.796+0000 7fd68ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.797+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0125f0 con 0x55ec9f376fe0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.797+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd69c012790 con 0x55ec9f376fe0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.797+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd694010430 con 0x55ec9f376fe0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.797+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd69c012990 con 0x55ec9f376fe0 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.797+0000 7fd6b2a53740 10 client.25660 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ec9efee210 con 0x55ec9f376fe0 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd694010430 con 0x55ec9f376fe0 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd69c013c30 con 0x55ec9f376fe0 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 10 client.25660 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 20 client.25660 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f392040 0x55ec9f3944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 4 client.25660 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.798+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec9f2e9f90 con 0x55ec9f392040 2024-04-26T22:06:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.799+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f392040 0x55ec9f3944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.799+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f392040 0x55ec9f3944c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78640ec9007ee4d6 server_cookie=a3734833bd47ee3c in_seq=0 out_seq=0 2024-04-26T22:06:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.799+0000 7fd68ea00640 10 client.25660 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.800+0000 7fd68ea00640 1 -- 172.21.15.12:0/4034587189 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec9f2e9f90 con 0x55ec9f392040 2024-04-26T22:06:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.801+0000 7fd68ea00640 10 client.25660 handle_command_reply: tid=0 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.801+0000 7fd6b2a53740 1 client.25660 shutdown 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.801+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 msgr2=0x55ec9f37f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.801+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d1e0 tx=0x7fd69c00d6a0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 shutdown_connections 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f392040 0x55ec9f3944c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4034587189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f37f0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd68ea00640 0 client.25660 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 >> 172.21.15.12:0/4034587189 conn(0x55ec9f3734f0 msgr2=0x55ec9f379270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 shutdown_connections 2024-04-26T22:06:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4034587189 wait complete. 2024-04-26T22:06:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.802+0000 7fd6b2a53740 20 client.25660 trim_cache size 0 max 16384 2024-04-26T22:06:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.831+0000 7fd6b2a53740 1 Processor -- start 2024-04-26T22:06:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.831+0000 7fd6b2a53740 1 -- start start 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 0x55ec9f38dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f37e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 0x55ec9f3790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd68800dea0 con 0x55ec9f3821c0 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd688010720 con 0x55ec9f376fe0 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.832+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd68800e510 con 0x55ec9f377900 2024-04-26T22:06:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.833+0000 7fd6adc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 0x55ec9f3790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.833+0000 7fd6adc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 0x55ec9f3790f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53018/0 (socket says 172.21.15.12:53018) 2024-04-26T22:06:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.834+0000 7fd6adc00640 1 -- 172.21.15.12:0/1243524386 learned_addr learned my addr 172.21.15.12:0/1243524386 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.834+0000 7fd6ad200640 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 0x55ec9f38dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.834+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f37e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.838+0000 7fd6ad200640 1 -- 172.21.15.12:0/1243524386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 msgr2=0x55ec9f37e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.838+0000 7fd6ad200640 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f37e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.838+0000 7fd6ad200640 1 -- 172.21.15.12:0/1243524386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 msgr2=0x55ec9f3790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.838+0000 7fd6ad200640 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 0x55ec9f3790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.838+0000 7fd6ad200640 1 -- 172.21.15.12:0/1243524386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9f377730 con 0x55ec9f376fe0 2024-04-26T22:06:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.839+0000 7fd6ad200640 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 0x55ec9f38dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d1e0 tx=0x7fd69c007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bbf39f07fee87c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.839+0000 7fd699600640 1 -- 172.21.15.12:0/1243524386 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00dba0 con 0x55ec9f376fe0 2024-04-26T22:06:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.839+0000 7fd699600640 1 -- 172.21.15.12:0/1243524386 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c02b070 con 0x55ec9f376fe0 2024-04-26T22:06:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.839+0000 7fd699600640 1 -- 172.21.15.12:0/1243524386 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c012d90 con 0x55ec9f376fe0 2024-04-26T22:06:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.841+0000 7fd6b2a53740 1 -- 172.21.15.12:0/1243524386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 msgr2=0x55ec9f38dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.841+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 0x55ec9f38dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d1e0 tx=0x7fd69c007bd0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.843+0000 7fd6b2a53740 1 -- 172.21.15.12:0/1243524386 shutdown_connections 2024-04-26T22:06:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.843+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f3821c0 0x55ec9f3790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.843+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f377900 0x55ec9f37e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.843+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/1243524386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f376fe0 0x55ec9f38dcb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.843+0000 7fd6b2a53740 1 -- 172.21.15.12:0/1243524386 >> 172.21.15.12:0/1243524386 conn(0x55ec9f3734f0 msgr2=0x55ec9f3789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.848+0000 7fd6b2a53740 1 -- 172.21.15.12:0/1243524386 shutdown_connections 2024-04-26T22:06:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.848+0000 7fd6b2a53740 1 -- 172.21.15.12:0/1243524386 wait complete. 2024-04-26T22:06:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.849+0000 7fd6b2a53740 1 -- start start 2024-04-26T22:06:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.852+0000 7fd6b2a53740 1 Processor -- start 2024-04-26T22:06:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.853+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f38a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd68800ebe0 con 0x55ec9f377900 2024-04-26T22:06:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd688010050 con 0x55ec9f3821c0 2024-04-26T22:06:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd68800f2b0 con 0x55ec9f376fe0 2024-04-26T22:06:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.855+0000 7fd6b2a53740 10 client.0 _ll_register_callbacks cb 0x7fd69c0071c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.856+0000 7fd6adc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.856+0000 7fd6adc00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:39658/0 (socket says 172.21.15.12:39658) 2024-04-26T22:06:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.856+0000 7fd6adc00640 1 -- 172.21.15.12:0/4271299511 learned_addr learned my addr 172.21.15.12:0/4271299511 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.856+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f38a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.857+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.857+0000 7fd6ad200640 1 -- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 msgr2=0x55ec9f37bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.857+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.857+0000 7fd6ad200640 1 -- 172.21.15.12:0/4271299511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 msgr2=0x55ec9f37b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd6ad200640 1 -- 172.21.15.12:0/4271299511 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9f0ba620 con 0x55ec9f376fe0 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd6adc00640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd6ad200640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f38a490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00a010 tx=0x7fd69c00dce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c07cda5787d7055 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.858+0000 7fd68ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:06:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.860+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c01d950 con 0x55ec9f376fe0 2024-04-26T22:06:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.861+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7fd69c01daf0 con 0x55ec9f376fe0 2024-04-26T22:06:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.862+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69c02b070 con 0x55ec9f376fe0 2024-04-26T22:06:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.863+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd69c01dcf0 con 0x55ec9f376fe0 2024-04-26T22:06:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.863+0000 7fd6b2a53740 10 client.25666 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.863+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ec9f296cb0 con 0x55ec9f376fe0 2024-04-26T22:06:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.864+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c02b070 con 0x55ec9f376fe0 2024-04-26T22:06:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.864+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd69c025070 con 0x55ec9f376fe0 2024-04-26T22:06:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.865+0000 7fd6b2a53740 10 client.25666 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.865+0000 7fd6b2a53740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.865+0000 7fd6b2a53740 20 client.25666 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.866+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f3f8730 0x55ec9f3fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.866+0000 7fd6b2a53740 4 client.25666 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:06:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.866+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec9f2e3e00 con 0x55ec9f3f8730 2024-04-26T22:06:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.866+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f3f8730 0x55ec9f3fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.869+0000 7fd6a7e00640 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f3f8730 0x55ec9f3fab20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e62fae72f0d094 server_cookie=ec6b02b81956b5d4 in_seq=0 out_seq=0 2024-04-26T22:06:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.869+0000 7fd68ea00640 10 client.25666 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd68ea00640 1 -- 172.21.15.12:0/4271299511 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec9f2e3e00 con 0x55ec9f3f8730 2024-04-26T22:06:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd68ea00640 10 client.25666 handle_command_reply: tid=0 2024-04-26T22:06:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd6b2a53740 1 client.25666 shutdown 2024-04-26T22:06:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 msgr2=0x55ec9f38a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f38a490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00a010 tx=0x7fd69c00dce0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 shutdown_connections 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.894+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55ec9f3f8730 0x55ec9f3fab20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec9f3821c0 0x55ec9f37bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec9f377900 0x55ec9f37b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 --2- 172.21.15.12:0/4271299511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec9f376fe0 0x55ec9f38a490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 >> 172.21.15.12:0/4271299511 conn(0x55ec9f3734f0 msgr2=0x55ec9f372650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd68ea00640 0 client.25666 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 shutdown_connections 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.895+0000 7fd6b2a53740 1 -- 172.21.15.12:0/4271299511 wait complete. 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.896+0000 7fd6b2a53740 20 client.25666 trim_cache size 0 max 16384 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:06:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:06:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.897+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 msgr2=0x7fd6900a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.897+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 0x7fd6900a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd694013830 tx=0x7fd694013cf0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 msgr2=0x7fd6a810d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a810d530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0006010 tx=0x7fd6a002f3b0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 shutdown_connections 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fd6900a3ee0 0x7fd6900a63a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6a8108760 0x7fd6a810d530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd6a8103110 0x7fd6a810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 --2- 172.21.15.12:0/1590052173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6a8102750 0x7fd6a810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 >> 172.21.15.12:0/1590052173 conn(0x7fd6a80fe280 msgr2=0x7fd6a80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 shutdown_connections 2024-04-26T22:06:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:10.898+0000 7fd6ae600640 1 -- 172.21.15.12:0/1590052173 wait complete. 2024-04-26T22:06:11.822 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:11 smithi012 ceph-mon[28184]: pgmap v110: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 85 op/s 2024-04-26T22:06:11.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:11 smithi154 ceph-mon[32578]: pgmap v110: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 85 op/s 2024-04-26T22:06:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:11 smithi077 ceph-mon[32280]: pgmap v110: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 85 op/s 2024-04-26T22:06:13.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:13 smithi012 ceph-mon[28184]: pgmap v111: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 344 MiB/s wr, 109 op/s 2024-04-26T22:06:13.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:13 smithi154 ceph-mon[32578]: pgmap v111: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 344 MiB/s wr, 109 op/s 2024-04-26T22:06:13.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:13 smithi077 ceph-mon[32280]: pgmap v111: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 344 MiB/s wr, 109 op/s 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 rewriters = 293870.34 kB/sec 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 rewriters = 293863.94 kB/sec 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 293870.34 kB/sec 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 293870.34 kB/sec 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 293870.34 kB/sec 2024-04-26T22:06:15.497 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 10485760.00 kB 2024-04-26T22:06:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:15 smithi012 ceph-mon[28184]: pgmap v112: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 313 MiB/s wr, 97 op/s 2024-04-26T22:06:15.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:15 smithi154 ceph-mon[32578]: pgmap v112: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 313 MiB/s wr, 97 op/s 2024-04-26T22:06:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:15 smithi077 ceph-mon[32280]: pgmap v112: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 313 MiB/s wr, 97 op/s 2024-04-26T22:06:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:17 smithi012 ceph-mon[28184]: pgmap v113: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 269 MiB/s wr, 72 op/s 2024-04-26T22:06:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:17 smithi154 ceph-mon[32578]: pgmap v113: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 269 MiB/s wr, 72 op/s 2024-04-26T22:06:17.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:17 smithi077 ceph-mon[32280]: pgmap v113: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 269 MiB/s wr, 72 op/s 2024-04-26T22:06:19.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:19 smithi012 ceph-mon[28184]: pgmap v114: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 364 MiB/s wr, 96 op/s 2024-04-26T22:06:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:19 smithi154 ceph-mon[32578]: pgmap v114: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 364 MiB/s wr, 96 op/s 2024-04-26T22:06:19.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:19 smithi077 ceph-mon[32280]: pgmap v114: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 364 MiB/s wr, 96 op/s 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 readers = 5731577.00 kB/sec 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 readers = 5728965.52 kB/sec 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 5731577.00 kB/sec 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 5731577.00 kB/sec 2024-04-26T22:06:21.538 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 5731577.00 kB/sec 2024-04-26T22:06:21.539 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 10485760.00 kB 2024-04-26T22:06:21.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:21 smithi012 ceph-mon[28184]: pgmap v115: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 MiB/s wr, 61 op/s 2024-04-26T22:06:21.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:21 smithi154 ceph-mon[32578]: pgmap v115: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 MiB/s wr, 61 op/s 2024-04-26T22:06:21.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:21 smithi077 ceph-mon[32280]: pgmap v115: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 MiB/s wr, 61 op/s 2024-04-26T22:06:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:23 smithi154 ceph-mon[32578]: pgmap v116: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 225 MiB/s wr, 62 op/s 2024-04-26T22:06:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:23 smithi077 ceph-mon[32280]: pgmap v116: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 225 MiB/s wr, 62 op/s 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 -- 172.21.15.12:0/136266763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8105500 msgr2=0x7f5ca81119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 --2- 172.21.15.12:0/136266763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8105500 0x7f5ca81119c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009c10 tx=0x7f5ca002a6e0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 -- 172.21.15.12:0/136266763 shutdown_connections 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 --2- 172.21.15.12:0/136266763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8105500 0x7f5ca81119c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 --2- 172.21.15.12:0/136266763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8104b40 0x7f5ca8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 --2- 172.21.15.12:0/136266763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 0x7f5ca810afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 -- 172.21.15.12:0/136266763 >> 172.21.15.12:0/136266763 conn(0x7f5ca8100630 msgr2=0x7f5ca8102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 -- 172.21.15.12:0/136266763 shutdown_connections 2024-04-26T22:06:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.050+0000 7f5cad200640 1 -- 172.21.15.12:0/136266763 wait complete. 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.051+0000 7f5cad200640 1 Processor -- start 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.051+0000 7f5cad200640 1 -- start start 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8105500 0x7f5ca819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 0x7f5ca819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8114df0 con 0x7f5ca8105500 2024-04-26T22:06:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8114c70 con 0x7f5ca810ac00 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5cad200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8114f70 con 0x7f5ca8104b40 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5ca5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8105500 0x7f5ca819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 0x7f5ca819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47460/0 (socket says 172.21.15.12:47460) 2024-04-26T22:06:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.052+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3876285323 learned_addr learned my addr 172.21.15.12:0/3876285323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3876285323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 msgr2=0x7f5ca819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 0x7f5ca819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8105500 msgr2=0x7f5ca819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8105500 0x7f5ca819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3876285323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca81a01a0 con 0x7f5ca8104b40 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.053+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e9f0 tx=0x7f5c9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.054+0000 7f5c9ea00640 1 -- 172.21.15.12:0/3876285323 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800cd80 con 0x7f5ca8104b40 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.054+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca81a0490 con 0x7f5ca8104b40 2024-04-26T22:06:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.054+0000 7f5c9ea00640 1 -- 172.21.15.12:0/3876285323 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c980108e0 con 0x7f5ca8104b40 2024-04-26T22:06:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.054+0000 7f5c9ea00640 1 -- 172.21.15.12:0/3876285323 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98018980 con 0x7f5ca8104b40 2024-04-26T22:06:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.054+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a7d20 con 0x7f5ca8104b40 2024-04-26T22:06:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.057+0000 7f5caff87740 1 Processor -- start 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 -- start start 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5c9ea00640 1 -- 172.21.15.12:0/3876285323 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5c98010430 con 0x7f5ca8104b40 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638b7e4e380 con 0x5638b7ee96f0 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638b7deac30 con 0x5638b7ef3fb0 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.058+0000 7f5caff87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5638b7e4a340 con 0x5638b7ee8dd0 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47466/0 (socket says 172.21.15.12:47466) 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca6800640 1 -- 172.21.15.12:0/864598853 learned_addr learned my addr 172.21.15.12:0/864598853 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 -- 172.21.15.12:0/864598853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 msgr2=0x5638b7ee91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 -- 172.21.15.12:0/864598853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 msgr2=0x5638b7ef3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 -- 172.21.15.12:0/864598853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b7e0c940 con 0x5638b7ef3fb0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca6800640 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.059+0000 7f5ca5400640 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef63a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009530 tx=0x7f5c90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a938ae10eed6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5c97400640 1 -- 172.21.15.12:0/864598853 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90004020 con 0x5638b7ef3fb0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5c97400640 1 -- 172.21.15.12:0/864598853 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c900041c0 con 0x5638b7ef3fb0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5c97400640 1 -- 172.21.15.12:0/864598853 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90019830 con 0x5638b7ef3fb0 2024-04-26T22:06:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5c9ea00640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 0x7f5c8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5c9ea00640 1 -- 172.21.15.12:0/3876285323 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5c980d0fd0 con 0x7f5ca8104b40 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5ca6800640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 0x7f5c8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5caff87740 1 -- 172.21.15.12:0/864598853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 msgr2=0x5638b7ef63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.060+0000 7f5caff87740 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef63a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009530 tx=0x7f5c90011810 comp rx=0 tx=0).stop 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 -- 172.21.15.12:0/864598853 shutdown_connections 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef63a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 --2- 172.21.15.12:0/864598853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee8dd0 0x5638b7ee91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 -- 172.21.15.12:0/864598853 >> 172.21.15.12:0/864598853 conn(0x5638b7ee52e0 msgr2=0x5638b7ee56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5ca6800640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 0x7f5c8c0a6280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0004af0 tx=0x7f5ca0005fd0 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 -- 172.21.15.12:0/864598853 shutdown_connections 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.061+0000 7f5caff87740 1 -- 172.21.15.12:0/864598853 wait complete. 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.062+0000 7f5caff87740 1 -- start start 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.062+0000 7f5caff87740 1 Processor -- start 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7ef1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638b7ef6be0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638b7dd5c60 con 0x5638b7ef3fb0 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5638b7ef6d60 con 0x5638b7ee96f0 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5caff87740 10 client.0 _ll_register_callbacks cb 0x5638b7e01e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40532/0 (socket says 172.21.15.12:40532) 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5ca5e00640 1 -- 172.21.15.12:0/2265311408 learned_addr learned my addr 172.21.15.12:0/2265311408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5ca5400640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7ef1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.063+0000 7f5ca6800640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 -- 172.21.15.12:0/2265311408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 msgr2=0x5638b7ef1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7ef1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 -- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 msgr2=0x5638b7ef1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 -- 172.21.15.12:0/2265311408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b7de4510 con 0x5638b7ee8dd0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5400640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7ef1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c9809c970 tx=0x7f5c980d4710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a02f7f469d52a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.064+0000 7f5c8be00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9809c5d0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5c980d4cd0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c900041c0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c98010720 con 0x5638b7ee8dd0 2024-04-26T22:06:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5caff87740 10 client.15543 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638b7b60210 con 0x5638b7ee8dd0 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.065+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c900041c0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5c980d3110 con 0x5638b7ee8dd0 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 10 client.15543 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 20 client.15543 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f03c40 0x5638b7f06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 4 client.15543 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b7e5bf90 con 0x5638b7f03c40 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.066+0000 7f5ca5400640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f03c40 0x5638b7f06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.067+0000 7f5c8be00640 10 client.15543 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.067+0000 7f5ca5400640 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f03c40 0x5638b7f06030 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d61c608069bc34a server_cookie=44a6a6ed1f98d486 in_seq=0 out_seq=0 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.068+0000 7f5c8be00640 1 -- 172.21.15.12:0/2265311408 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638b7e5bf90 con 0x5638b7f03c40 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.068+0000 7f5c8be00640 10 client.15543 handle_command_reply: tid=0 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.068+0000 7f5caff87740 1 client.15543 shutdown 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 msgr2=0x5638b7ef0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c9809c970 tx=0x7f5c980d4710 comp rx=0 tx=0).stop 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 shutdown_connections 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f03c40 0x5638b7f06030 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ef3fb0 0x5638b7ef1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7ef1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 --2- 172.21.15.12:0/2265311408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee8dd0 0x5638b7ef0ec0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5c8be00640 0 client.15543 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 >> 172.21.15.12:0/2265311408 conn(0x5638b7ee52e0 msgr2=0x5638b7eeb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 shutdown_connections 2024-04-26T22:06:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 1 -- 172.21.15.12:0/2265311408 wait complete. 2024-04-26T22:06:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.069+0000 7f5caff87740 20 client.15543 trim_cache size 0 max 16384 2024-04-26T22:06:24.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:23 smithi012 ceph-mon[28184]: pgmap v116: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 225 MiB/s wr, 62 op/s 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.087+0000 7f5caff87740 1 Processor -- start 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 -- start start 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 0x5638b7f0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81afde0 con 0x5638b7ee96f0 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81b2660 con 0x5638b7ee8dd0 2024-04-26T22:06:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5caff87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81b0450 con 0x5638b7ef3fb0 2024-04-26T22:06:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 0x5638b7f0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:59494/0 (socket says 172.21.15.12:59494) 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 0x5638b7f0b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47474/0 (socket says 172.21.15.12:47474) 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.088+0000 7f5ca5e00640 1 -- 172.21.15.12:0/1627515912 learned_addr learned my addr 172.21.15.12:0/1627515912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 -- 172.21.15.12:0/1627515912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 msgr2=0x5638b7f0b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 0x5638b7f0b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 -- 172.21.15.12:0/1627515912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 msgr2=0x5638b7ef07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 -- 172.21.15.12:0/1627515912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b7ee9520 con 0x5638b7ee8dd0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c980ce970 tx=0x7f5c98018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be49bfdbbe81d9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5c96a00640 1 -- 172.21.15.12:0/1627515912 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9809c5d0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5c96a00640 1 -- 172.21.15.12:0/1627515912 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c980a0f70 con 0x5638b7ee8dd0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.089+0000 7f5c96a00640 1 -- 172.21.15.12:0/1627515912 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98098f10 con 0x5638b7ee8dd0 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 -- 172.21.15.12:0/1627515912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 msgr2=0x5638b7effaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c980ce970 tx=0x7f5c98018420 comp rx=0 tx=0).stop 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 -- 172.21.15.12:0/1627515912 shutdown_connections 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ef3fb0 0x5638b7f0b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ee96f0 0x5638b7ef07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 --2- 172.21.15.12:0/1627515912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7effaa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 -- 172.21.15.12:0/1627515912 >> 172.21.15.12:0/1627515912 conn(0x5638b7ee52e0 msgr2=0x5638b7ee4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 -- 172.21.15.12:0/1627515912 shutdown_connections 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.090+0000 7f5caff87740 1 -- 172.21.15.12:0/1627515912 wait complete. 2024-04-26T22:06:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.091+0000 7f5caff87740 1 -- start start 2024-04-26T22:06:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.091+0000 7f5caff87740 1 Processor -- start 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7eee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ef3fb0 0x5638b7eee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81b0b20 con 0x5638b7ef3fb0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81b1f90 con 0x5638b7ee8dd0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81b11f0 con 0x5638b7ee96f0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5caff87740 10 client.0 _ll_register_callbacks cb 0x7f5c980d4810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ef3fb0 0x5638b7eee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7eee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:59498/0 (socket says 172.21.15.12:59498) 2024-04-26T22:06:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3190793126 learned_addr learned my addr 172.21.15.12:0/3190793126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3190793126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 msgr2=0x5638b7eee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7eee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3190793126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ef3fb0 msgr2=0x5638b7eee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ef3fb0 0x5638b7eee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5e00640 1 -- 172.21.15.12:0/3190793126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b7c2c620 con 0x5638b7ee8dd0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.092+0000 7f5ca5400640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7eee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5ca5e00640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c98010d20 tx=0x7f5c980d4e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cacbed6c7974875f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5c8be00640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980992e0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f5c9809c2c0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c980a0f70 con 0x5638b7ee8dd0 2024-04-26T22:06:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c980aa050 con 0x5638b7ee8dd0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5caff87740 10 client.25688 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5638b7e08cb0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.093+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c980a0f70 con 0x5638b7ee8dd0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5c980ceac0 con 0x5638b7ee8dd0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 10 client.25688 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 20 client.25688 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f6a7d0 0x5638b7f6cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 4 client.25688 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638b7e55e00 con 0x5638b7f6a7d0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.094+0000 7f5ca5400640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f6a7d0 0x5638b7f6cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.095+0000 7f5ca5400640 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f6a7d0 0x5638b7f6cbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c6244c2dda0f17 server_cookie=d22494baf6b21719 in_seq=0 out_seq=0 2024-04-26T22:06:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.095+0000 7f5c8be00640 10 client.25688 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5c8be00640 1 -- 172.21.15.12:0/3190793126 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638b7e55e00 con 0x5638b7f6a7d0 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5c8be00640 10 client.25688 handle_command_reply: tid=0 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 client.25688 shutdown 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 msgr2=0x5638b7eedde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c98010d20 tx=0x7f5c980d4e80 comp rx=0 tx=0).stop 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 shutdown_connections 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x5638b7f6a7d0 0x5638b7f6cbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638b7ef3fb0 0x5638b7eee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5638b7ee96f0 0x5638b7eee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 --2- 172.21.15.12:0/3190793126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b7ee8dd0 0x5638b7eedde0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5c8be00640 0 client.25688 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 >> 172.21.15.12:0/3190793126 conn(0x5638b7ee52e0 msgr2=0x5638b7ee3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 shutdown_connections 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 1 -- 172.21.15.12:0/3190793126 wait complete. 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.104+0000 7f5caff87740 20 client.25688 trim_cache size 0 max 16384 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:06:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:06:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:06:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 msgr2=0x7f5c8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 0x7f5c8c0a6280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0004af0 tx=0x7f5ca0005fd0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 msgr2=0x7f5ca819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e9f0 tx=0x7f5c9800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 shutdown_connections 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c8c0a3dc0 0x7f5c8c0a6280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca810ac00 0x7f5ca819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ca8105500 0x7f5ca819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 --2- 172.21.15.12:0/3876285323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ca8104b40 0x7f5ca819b1b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 >> 172.21.15.12:0/3876285323 conn(0x7f5ca8100630 msgr2=0x7f5ca8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 shutdown_connections 2024-04-26T22:06:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:24.105+0000 7f5cad200640 1 -- 172.21.15.12:0/3876285323 wait complete. 2024-04-26T22:06:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:25 smithi154 ceph-mon[32578]: pgmap v117: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 141 MiB/s wr, 37 op/s 2024-04-26T22:06:25.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:25 smithi077 ceph-mon[32280]: pgmap v117: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 141 MiB/s wr, 37 op/s 2024-04-26T22:06:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:25 smithi012 ceph-mon[28184]: pgmap v117: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 141 MiB/s wr, 37 op/s 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Children see throughput for 1 re-readers = 3774495.75 kB/sec 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Parent sees throughput for 1 re-readers = 3773440.33 kB/sec 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Min throughput per process = 3774495.75 kB/sec 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Max throughput per process = 3774495.75 kB/sec 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Avg throughput per process = 3774495.75 kB/sec 2024-04-26T22:06:26.418 INFO:tasks.workunit.client.0.smithi012.stdout: Min xfer = 10485760.00 kB 2024-04-26T22:06:27.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:27 smithi154 ceph-mon[32578]: pgmap v118: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 95 MiB/s wr, 25 op/s 2024-04-26T22:06:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:27 smithi077 ceph-mon[32280]: pgmap v118: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 95 MiB/s wr, 25 op/s 2024-04-26T22:06:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:27 smithi012 ceph-mon[28184]: pgmap v118: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 95 MiB/s wr, 25 op/s 2024-04-26T22:06:29.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:29 smithi154 ceph-mon[32578]: pgmap v119: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 95 MiB/s wr, 28 op/s 2024-04-26T22:06:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:29 smithi077 ceph-mon[32280]: pgmap v119: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 95 MiB/s wr, 28 op/s 2024-04-26T22:06:30.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:29 smithi012 ceph-mon[28184]: pgmap v119: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 95 MiB/s wr, 28 op/s 2024-04-26T22:06:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:31 smithi154 ceph-mon[32578]: pgmap v120: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-26T22:06:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:31 smithi077 ceph-mon[32280]: pgmap v120: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-26T22:06:32.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:31 smithi012 ceph-mon[28184]: pgmap v120: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-26T22:06:33.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:33 smithi154 ceph-mon[32578]: pgmap v121: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-04-26T22:06:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:33 smithi077 ceph-mon[32280]: pgmap v121: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-04-26T22:06:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:33 smithi012 ceph-mon[28184]: pgmap v121: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-04-26T22:06:35.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:35 smithi154 ceph-mon[32578]: pgmap v122: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:35.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:35 smithi077 ceph-mon[32280]: pgmap v122: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:36.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:35 smithi012 ceph-mon[28184]: pgmap v122: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:36.462 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:36.462 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:36.462 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-04-26T22:06:36.462 INFO:tasks.workunit.client.0.smithi012.stdout:iozone test complete. 2024-04-26T22:06:36.468 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T22:06:36.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T22:06:36.619 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-26T22:06:36.620 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T22:06:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 -- 172.21.15.12:0/55056001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 msgr2=0x7f8324101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 --2- 172.21.15.12:0/55056001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f8324101ac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402a480 comp rx=0 tx=0).stop 2024-04-26T22:06:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 -- 172.21.15.12:0/55056001 shutdown_connections 2024-04-26T22:06:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 --2- 172.21.15.12:0/55056001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f8324106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 --2- 172.21.15.12:0/55056001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 0x7f83241064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 --2- 172.21.15.12:0/55056001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f8324101ac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 -- 172.21.15.12:0/55056001 >> 172.21.15.12:0/55056001 conn(0x7f83240fd150 msgr2=0x7f83240ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.302+0000 7f8329200640 1 -- 172.21.15.12:0/55056001 shutdown_connections 2024-04-26T22:06:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.303+0000 7f8329200640 1 -- 172.21.15.12:0/55056001 wait complete. 2024-04-26T22:06:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.304+0000 7f8329200640 1 Processor -- start 2024-04-26T22:06:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 -- start start 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 0x7f832419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f83241a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8324119000 con 0x7f8324102000 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8324118e80 con 0x7f83241016e0 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8329200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8324119180 con 0x7f8324106ae0 2024-04-26T22:06:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8323400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8323400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:38696/0 (socket says 172.21.15.12:38696) 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8323400640 1 -- 172.21.15.12:0/3904816448 learned_addr learned my addr 172.21.15.12:0/3904816448 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.305+0000 7f8323e00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f83241a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323400640 1 -- 172.21.15.12:0/3904816448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 msgr2=0x7f83241a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323400640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f83241a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323400640 1 -- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 msgr2=0x7f832419fb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8322a00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 0x7f832419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323400640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 0x7f832419fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323400640 1 -- 172.21.15.12:0/3904816448 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a45b0 con 0x7f83241016e0 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.306+0000 7f8323e00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f83241a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.307+0000 7f8323400640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8314005ab0 tx=0x7f8314032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.308+0000 7f831be00640 1 -- 172.21.15.12:0/3904816448 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831403f030 con 0x7f83241016e0 2024-04-26T22:06:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.308+0000 7f831be00640 1 -- 172.21.15.12:0/3904816448 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314003cb0 con 0x7f83241016e0 2024-04-26T22:06:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.308+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a4840 con 0x7f83241016e0 2024-04-26T22:06:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.308+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241ac130 con 0x7f83241016e0 2024-04-26T22:06:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.309+0000 7f831be00640 1 -- 172.21.15.12:0/3904816448 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314035070 con 0x7f83241016e0 2024-04-26T22:06:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.318+0000 7f831be00640 1 -- 172.21.15.12:0/3904816448 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8314042030 con 0x7f83241016e0 2024-04-26T22:06:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.319+0000 7f832d585740 1 Processor -- start 2024-04-26T22:06:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 -- start start 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55928d552380 con 0x55928d5ed900 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55928d4eec30 con 0x55928d5f81c0 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f832d585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55928d54e340 con 0x55928d5ecfe0 2024-04-26T22:06:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.320+0000 7f831be00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 0x7f83080a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.321+0000 7f8323400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.321+0000 7f8323400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57248/0 (socket says 172.21.15.12:57248) 2024-04-26T22:06:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.321+0000 7f8323400640 1 -- 172.21.15.12:0/1003177818 learned_addr learned my addr 172.21.15.12:0/1003177818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.321+0000 7f831be00640 1 -- 172.21.15.12:0/3904816448 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f83140f0ac0 con 0x7f83241016e0 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.321+0000 7f8322a00640 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323e00640 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323400640 1 -- 172.21.15.12:0/1003177818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 msgr2=0x55928d5ed3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323400640 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ed3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323400640 1 -- 172.21.15.12:0/1003177818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 msgr2=0x55928d5fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323400640 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323e00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 0x7f83080a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.322+0000 7f8323400640 1 -- 172.21.15.12:0/1003177818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d510940 con 0x55928d5ed900 2024-04-26T22:06:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.323+0000 7f8323400640 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8314003fd0 tx=0x7f83140bc540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa817646a9308ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.323+0000 7f8323e00640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 0x7f83080a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b910 tx=0x7f831c009650 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.324+0000 7f8318c00640 1 -- 172.21.15.12:0/1003177818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314049070 con 0x55928d5ed900 2024-04-26T22:06:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.324+0000 7f8318c00640 1 -- 172.21.15.12:0/1003177818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140c61e0 con 0x55928d5ed900 2024-04-26T22:06:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.324+0000 7f8318c00640 1 -- 172.21.15.12:0/1003177818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140bdc10 con 0x55928d5ed900 2024-04-26T22:06:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.325+0000 7f832d585740 1 -- 172.21.15.12:0/1003177818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 msgr2=0x55928d5f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.325+0000 7f832d585740 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8314003fd0 tx=0x7f83140bc540 comp rx=0 tx=0).stop 2024-04-26T22:06:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 -- 172.21.15.12:0/1003177818 shutdown_connections 2024-04-26T22:06:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ed900 0x55928d5f7c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 --2- 172.21.15.12:0/1003177818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ed3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 -- 172.21.15.12:0/1003177818 >> 172.21.15.12:0/1003177818 conn(0x55928d5e94f0 msgr2=0x55928d5e98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 -- 172.21.15.12:0/1003177818 shutdown_connections 2024-04-26T22:06:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.326+0000 7f832d585740 1 -- 172.21.15.12:0/1003177818 wait complete. 2024-04-26T22:06:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.327+0000 7f832d585740 1 -- start start 2024-04-26T22:06:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.327+0000 7f832d585740 1 Processor -- start 2024-04-26T22:06:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 0x55928d5f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55928d5fabb0 con 0x55928d5ecfe0 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55928d4d9c60 con 0x55928d5f81c0 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55928d5fad30 con 0x55928d5ed900 2024-04-26T22:06:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.329+0000 7f832d585740 10 client.0 _ll_register_callbacks cb 0x55928d505e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.330+0000 7f8322a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.330+0000 7f8322a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:52492/0 (socket says 172.21.15.12:52492) 2024-04-26T22:06:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.330+0000 7f8322a00640 1 -- 172.21.15.12:0/1215328034 learned_addr learned my addr 172.21.15.12:0/1215328034 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.330+0000 7f8323e00640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.330+0000 7f8323400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 0x55928d5f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.331+0000 7f8322a00640 1 -- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 msgr2=0x55928d5f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.331+0000 7f8322a00640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.331+0000 7f8322a00640 1 -- 172.21.15.12:0/1215328034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 msgr2=0x55928d5f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.331+0000 7f8322a00640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 0x55928d5f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.331+0000 7f8322a00640 1 -- 172.21.15.12:0/1215328034 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928d4e8510 con 0x55928d5ed900 2024-04-26T22:06:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.332+0000 7f8323e00640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.332+0000 7f8323400640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 0x55928d5f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.333+0000 7f8322a00640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8310013710 tx=0x7f8310013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe015a1e1da965d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.333+0000 7f830d600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:06:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.334+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310011dd0 con 0x55928d5ed900 2024-04-26T22:06:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.334+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83140c61e0 con 0x55928d5ed900 2024-04-26T22:06:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.335+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f8310011bd0 con 0x55928d5ed900 2024-04-26T22:06:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.335+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8310011dd0 con 0x55928d5ed900 2024-04-26T22:06:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.335+0000 7f832d585740 10 client.25693 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.335+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55928d264210 con 0x55928d5ed900 2024-04-26T22:06:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.336+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140c61e0 con 0x55928d5ed900 2024-04-26T22:06:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f831001a070 con 0x55928d5ed900 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 10 client.25693 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 20 client.25693 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d607c50 0x55928d60a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 4 client.25693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928d55ff90 con 0x55928d607c50 2024-04-26T22:06:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.337+0000 7f8323400640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d607c50 0x55928d60a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.338+0000 7f830d600640 10 client.25693 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.338+0000 7f8323400640 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d607c50 0x55928d60a0a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb904c2f0eefdb5 server_cookie=68fc1260564623ae in_seq=0 out_seq=0 2024-04-26T22:06:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.339+0000 7f830d600640 1 -- 172.21.15.12:0/1215328034 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55928d55ff90 con 0x55928d607c50 2024-04-26T22:06:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.339+0000 7f830d600640 10 client.25693 handle_command_reply: tid=0 2024-04-26T22:06:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 client.25693 shutdown 2024-04-26T22:06:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 msgr2=0x55928d5f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8310013710 tx=0x7f8310013bd0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 shutdown_connections 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d607c50 0x55928d60a0a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5f81c0 0x55928d5f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ed900 0x55928d5f55f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 --2- 172.21.15.12:0/1215328034 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5ecfe0 0x55928d5f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 >> 172.21.15.12:0/1215328034 conn(0x55928d5e94f0 msgr2=0x55928d5ef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.340+0000 7f830d600640 0 client.25693 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.341+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 shutdown_connections 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.341+0000 7f832d585740 1 -- 172.21.15.12:0/1215328034 wait complete. 2024-04-26T22:06:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.341+0000 7f832d585740 20 client.25693 trim_cache size 0 max 16384 2024-04-26T22:06:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 Processor -- start 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 -- start start 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d60fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d60ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f830400df00 con 0x55928d5f81c0 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8304010780 con 0x55928d5ed900 2024-04-26T22:06:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.374+0000 7f832d585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f830400e570 con 0x55928d5ecfe0 2024-04-26T22:06:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.375+0000 7f8323400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.375+0000 7f8323400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:52506/0 (socket says 172.21.15.12:52506) 2024-04-26T22:06:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.375+0000 7f8323400640 1 -- 172.21.15.12:0/4134334857 learned_addr learned my addr 172.21.15.12:0/4134334857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.375+0000 7f8322a00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d60fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.375+0000 7f8323e00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d60ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 -- 172.21.15.12:0/4134334857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 msgr2=0x55928d5ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 -- 172.21.15.12:0/4134334857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 msgr2=0x55928d60ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d60ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 -- 172.21.15.12:0/4134334857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d5ed730 con 0x55928d5ed900 2024-04-26T22:06:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8323e00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d60ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8323400640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.376+0000 7f8322a00640 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d60fa00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831000e670 tx=0x7f8310009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78626cc92a20527a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.377+0000 7f830fe00640 1 -- 172.21.15.12:0/4134334857 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310030070 con 0x55928d5ed900 2024-04-26T22:06:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.377+0000 7f830fe00640 1 -- 172.21.15.12:0/4134334857 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831000e3d0 con 0x55928d5ed900 2024-04-26T22:06:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.377+0000 7f830fe00640 1 -- 172.21.15.12:0/4134334857 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831002ba50 con 0x55928d5ed900 2024-04-26T22:06:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 -- 172.21.15.12:0/4134334857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 msgr2=0x55928d60fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d60fa00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831000e670 tx=0x7f8310009320 comp rx=0 tx=0).stop 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 -- 172.21.15.12:0/4134334857 shutdown_connections 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d60ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d60fa00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 --2- 172.21.15.12:0/4134334857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 -- 172.21.15.12:0/4134334857 >> 172.21.15.12:0/4134334857 conn(0x55928d5e94f0 msgr2=0x55928d5f0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 -- 172.21.15.12:0/4134334857 shutdown_connections 2024-04-26T22:06:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.378+0000 7f832d585740 1 -- 172.21.15.12:0/4134334857 wait complete. 2024-04-26T22:06:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.379+0000 7f832d585740 1 -- start start 2024-04-26T22:06:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.379+0000 7f832d585740 1 Processor -- start 2024-04-26T22:06:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.380+0000 7f832d585740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.380+0000 7f832d585740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d6003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.380+0000 7f8323400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f8323400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:52516/0 (socket says 172.21.15.12:52516) 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f832d585740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d669580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f832d585740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f830400ec40 con 0x55928d5f81c0 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f832d585740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83040100b0 con 0x55928d5ed900 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f832d585740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f830400f310 con 0x55928d5ecfe0 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f832d585740 10 client.0 _ll_register_callbacks cb 0x7f8310009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f8323400640 1 -- 172.21.15.12:0/754973201 learned_addr learned my addr 172.21.15.12:0/754973201 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:06:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.381+0000 7f8323e00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d669580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.382+0000 7f8322a00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d6003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.382+0000 7f8323400640 1 -- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 msgr2=0x55928d6003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.382+0000 7f8323400640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d6003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.382+0000 7f8323400640 1 -- 172.21.15.12:0/754973201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 msgr2=0x55928d669580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.383+0000 7f8322a00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d6003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.383+0000 7f8323400640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d669580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.383+0000 7f8323400640 1 -- 172.21.15.12:0/754973201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928d330620 con 0x55928d5ecfe0 2024-04-26T22:06:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.383+0000 7f8323e00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d669580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.383+0000 7f8323400640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83140bc3b0 tx=0x7f83140bc5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=699d1d5eed73cf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f830d600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314049040 con 0x55928d5ecfe0 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9459+0+0 (secure 0 0 0) 0x7f83140ee0e0 con 0x55928d5ecfe0 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831000e3d0 con 0x55928d5ecfe0 2024-04-26T22:06:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f83140c0a60 con 0x55928d5ecfe0 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f832d585740 10 client.25699 fetch_fsmap learned FSMap version 21 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.384+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55928d50ccb0 con 0x55928d5ecfe0 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831000e3d0 con 0x55928d5ecfe0 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f83140c4e90 con 0x55928d5ecfe0 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 10 client.25699 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 20 client.25699 populate_metadata read hostname 'smithi012' 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d67f280 0x55928d681670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 4 client.25699 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T22:06:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 --> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55928d559e00 con 0x55928d67f280 2024-04-26T22:06:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.385+0000 7f8322a00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d67f280 0x55928d681670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.386+0000 7f8322a00640 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d67f280 0x55928d681670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14aeb18cdf5ad0a0 server_cookie=3dc2f76e1ef9d418 in_seq=0 out_seq=0 2024-04-26T22:06:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.386+0000 7f830d600640 10 client.25699 ms_handle_connect on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.402+0000 7f830d600640 1 -- 172.21.15.12:0/754973201 <== mds.0 v2:172.21.15.77:6838/4217479283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55928d559e00 con 0x55928d67f280 2024-04-26T22:06:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.402+0000 7f830d600640 10 client.25699 handle_command_reply: tid=0 2024-04-26T22:06:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.402+0000 7f832d585740 1 client.25699 shutdown 2024-04-26T22:06:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 msgr2=0x55928d5ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83140bc3b0 tx=0x7f83140bc5b0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 shutdown_connections 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:6838/4217479283,v1:172.21.15.77:6839/4217479283] conn(0x55928d67f280 0x55928d681670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55928d5f81c0 0x55928d669580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55928d5ed900 0x55928d6003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 --2- 172.21.15.12:0/754973201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55928d5ecfe0 0x55928d5ffe90 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83140bc3b0 tx=0x7f83140bc5b0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 >> 172.21.15.12:0/754973201 conn(0x55928d5e94f0 msgr2=0x55928d5f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f830d600640 0 client.25699 ms_handle_reset on v2:172.21.15.77:6838/4217479283 2024-04-26T22:06:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 shutdown_connections 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 1 -- 172.21.15.12:0/754973201 wait complete. 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.403+0000 7f832d585740 20 client.25699 trim_cache size 0 max 16384 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-26T22:06:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-26T22:06:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 msgr2=0x7f83080a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 0x7f83080a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b910 tx=0x7f831c009650 comp rx=0 tx=0).stop 2024-04-26T22:06:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 msgr2=0x7f832419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8314005ab0 tx=0x7f8314032c80 comp rx=0 tx=0).stop 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 shutdown_connections 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f83080a3e60 0x7f83080a6320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324106ae0 0x7f83241a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8324102000 0x7f832419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 --2- 172.21.15.12:0/3904816448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83241016e0 0x7f832419f5c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.408+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 >> 172.21.15.12:0/3904816448 conn(0x7f83240fd150 msgr2=0x7f83240fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.409+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 shutdown_connections 2024-04-26T22:06:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-26T22:06:37.409+0000 7f8329200640 1 -- 172.21.15.12:0/3904816448 wait complete. 2024-04-26T22:06:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:37 smithi154 ceph-mon[32578]: pgmap v123: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:37.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:37 smithi077 ceph-mon[32280]: pgmap v123: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:38.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:37 smithi012 ceph-mon[28184]: pgmap v123: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.4 KiB/s wr, 4 op/s 2024-04-26T22:06:38.114 DEBUG:teuthology.parallel:result is None 2024-04-26T22:06:38.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:06:38.201 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:06:38.201 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T22:06:38.217 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:06:38.231 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T22:06:38.242 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T22:06:38.242 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T22:06:38.242 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:06:38.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T22:06:38.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T22:06:38.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:38 smithi154 ceph-mon[32578]: pgmap v124: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.6 KiB/s wr, 210 op/s 2024-04-26T22:06:38.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:38 smithi077 ceph-mon[32280]: pgmap v124: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.6 KiB/s wr, 210 op/s 2024-04-26T22:06:39.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:38 smithi012 ceph-mon[28184]: pgmap v124: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.6 KiB/s wr, 210 op/s 2024-04-26T22:06:41.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:41 smithi154 ceph-mon[32578]: pgmap v125: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 206 op/s 2024-04-26T22:06:41.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:41 smithi077 ceph-mon[32280]: pgmap v125: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 206 op/s 2024-04-26T22:06:41.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:41 smithi012 ceph-mon[28184]: pgmap v125: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.2 KiB/s wr, 206 op/s 2024-04-26T22:06:43.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:43 smithi154 ceph-mon[32578]: pgmap v126: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 216 op/s 2024-04-26T22:06:43.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:43 smithi077 ceph-mon[32280]: pgmap v126: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 216 op/s 2024-04-26T22:06:43.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:43 smithi012 ceph-mon[28184]: pgmap v126: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 216 op/s 2024-04-26T22:06:45.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:45 smithi154 ceph-mon[32578]: pgmap v127: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.6 KiB/s wr, 219 op/s 2024-04-26T22:06:45.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:45 smithi077 ceph-mon[32280]: pgmap v127: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.6 KiB/s wr, 219 op/s 2024-04-26T22:06:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:45 smithi012 ceph-mon[28184]: pgmap v127: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.6 KiB/s wr, 219 op/s 2024-04-26T22:06:47.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:47 smithi077 ceph-mon[32280]: pgmap v128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 219 op/s 2024-04-26T22:06:47.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:47 smithi012 ceph-mon[28184]: pgmap v128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 219 op/s 2024-04-26T22:06:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:47 smithi154 ceph-mon[32578]: pgmap v128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.5 KiB/s wr, 219 op/s 2024-04-26T22:06:49.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:49 smithi077 ceph-mon[32280]: pgmap v129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 KiB/s wr, 220 op/s 2024-04-26T22:06:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:49 smithi012 ceph-mon[28184]: pgmap v129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 KiB/s wr, 220 op/s 2024-04-26T22:06:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:49 smithi154 ceph-mon[32578]: pgmap v129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 KiB/s wr, 220 op/s 2024-04-26T22:06:51.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:51 smithi077 ceph-mon[32280]: pgmap v130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T22:06:51.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:51 smithi012 ceph-mon[28184]: pgmap v130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T22:06:51.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:51 smithi154 ceph-mon[32578]: pgmap v130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T22:06:52.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:52 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:52.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:52 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:52.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:52 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:53.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:53 smithi077 ceph-mon[32280]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-26T22:06:53.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:53 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:53.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:53 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:53.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:53 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:06:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:53 smithi012 ceph-mon[28184]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-26T22:06:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:53 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:53 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:53 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:06:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:53 smithi154 ceph-mon[32578]: pgmap v131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-26T22:06:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:53 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:53.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:53 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:53.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:53 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:06:55.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:55 smithi077 ceph-mon[32280]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T22:06:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:55 smithi012 ceph-mon[28184]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T22:06:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:55 smithi154 ceph-mon[32578]: pgmap v132: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T22:06:57.442 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:57 smithi154 ceph-mon[32578]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-26T22:06:57.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:57 smithi077 ceph-mon[32280]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-26T22:06:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:57 smithi012 ceph-mon[28184]: pgmap v133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-26T22:06:59.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:06:59 smithi077 ceph-mon[32280]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T22:06:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:06:59 smithi012 ceph-mon[28184]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T22:06:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:06:59 smithi154 ceph-mon[32578]: pgmap v134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T22:07:01.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:01 smithi077 ceph-mon[32280]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:01 smithi012 ceph-mon[28184]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:01.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:01 smithi154 ceph-mon[32578]: pgmap v135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:03.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:03 smithi077 ceph-mon[32280]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-26T22:07:03.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:03 smithi012 ceph-mon[28184]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-26T22:07:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:03 smithi154 ceph-mon[32578]: pgmap v136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 12 KiB/s wr, 4 op/s 2024-04-26T22:07:05.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:05 smithi077 ceph-mon[32280]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:05.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:05 smithi012 ceph-mon[28184]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:05 smithi154 ceph-mon[32578]: pgmap v137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:07.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:07 smithi077 ceph-mon[32280]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:07 smithi012 ceph-mon[28184]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:07 smithi154 ceph-mon[32578]: pgmap v138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:09.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:09 smithi077 ceph-mon[32280]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:09 smithi012 ceph-mon[28184]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:09 smithi154 ceph-mon[32578]: pgmap v139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:11.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:11 smithi077 ceph-mon[32280]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:11 smithi012 ceph-mon[28184]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:11 smithi154 ceph-mon[32578]: pgmap v140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:07:13.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:13 smithi077 ceph-mon[32280]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:13 smithi012 ceph-mon[28184]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:13 smithi154 ceph-mon[32578]: pgmap v141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:07:15.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:15 smithi077 ceph-mon[32280]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T22:07:15.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:15 smithi012 ceph-mon[28184]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T22:07:15.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:15 smithi154 ceph-mon[32578]: pgmap v142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T22:07:17.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:17 smithi012 ceph-mon[28184]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:07:17.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:17 smithi154 ceph-mon[32578]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:07:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:17 smithi077 ceph-mon[32280]: pgmap v143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:07:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:19 smithi012 ceph-mon[28184]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:07:19.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:19 smithi154 ceph-mon[32578]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:07:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:19 smithi077 ceph-mon[32280]: pgmap v144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:07:21.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:21 smithi012 ceph-mon[28184]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:07:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:21 smithi154 ceph-mon[32578]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:07:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:21 smithi077 ceph-mon[32280]: pgmap v145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:07:23.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:23 smithi012 ceph-mon[28184]: pgmap v146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:07:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:23 smithi154 ceph-mon[32578]: pgmap v146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:07:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:23 smithi077 ceph-mon[32280]: pgmap v146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:07:25.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:25 smithi012 ceph-mon[28184]: pgmap v147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:25 smithi154 ceph-mon[32578]: pgmap v147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:25 smithi077 ceph-mon[32280]: pgmap v147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:27.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:27 smithi012 ceph-mon[28184]: pgmap v148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:27.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:27 smithi154 ceph-mon[32578]: pgmap v148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:27 smithi077 ceph-mon[32280]: pgmap v148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:29.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:29 smithi012 ceph-mon[28184]: pgmap v149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:29.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:29 smithi154 ceph-mon[32578]: pgmap v149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:29 smithi077 ceph-mon[32280]: pgmap v149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:31.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:31 smithi012 ceph-mon[28184]: pgmap v150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:31.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:31 smithi154 ceph-mon[32578]: pgmap v150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:31 smithi077 ceph-mon[32280]: pgmap v150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:33.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:33 smithi012 ceph-mon[28184]: pgmap v151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:33.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:33 smithi154 ceph-mon[32578]: pgmap v151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:33 smithi077 ceph-mon[32280]: pgmap v151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:35.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:35 smithi012 ceph-mon[28184]: pgmap v152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:35.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:35 smithi154 ceph-mon[32578]: pgmap v152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:35.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:35 smithi077 ceph-mon[32280]: pgmap v152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:37.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:37 smithi012 ceph-mon[28184]: pgmap v153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:37.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:37 smithi154 ceph-mon[32578]: pgmap v153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:37.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:37 smithi077 ceph-mon[32280]: pgmap v153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:39.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:39 smithi012 ceph-mon[28184]: pgmap v154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:39.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:39 smithi154 ceph-mon[32578]: pgmap v154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:39 smithi077 ceph-mon[32280]: pgmap v154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:41.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:41 smithi012 ceph-mon[28184]: pgmap v155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:41.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:41 smithi154 ceph-mon[32578]: pgmap v155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:41 smithi077 ceph-mon[32280]: pgmap v155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:43.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:43 smithi012 ceph-mon[28184]: pgmap v156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:43.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:43 smithi154 ceph-mon[32578]: pgmap v156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:43 smithi077 ceph-mon[32280]: pgmap v156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:45 smithi012 ceph-mon[28184]: pgmap v157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:45.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:45 smithi154 ceph-mon[32578]: pgmap v157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:45 smithi077 ceph-mon[32280]: pgmap v157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:47.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:47 smithi012 ceph-mon[28184]: pgmap v158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:47 smithi154 ceph-mon[32578]: pgmap v158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:47.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:47 smithi077 ceph-mon[32280]: pgmap v158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:49 smithi012 ceph-mon[28184]: pgmap v159: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:49 smithi154 ceph-mon[32578]: pgmap v159: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:49 smithi077 ceph-mon[32280]: pgmap v159: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:51.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:51 smithi154 ceph-mon[32578]: pgmap v160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:51 smithi077 ceph-mon[32280]: pgmap v160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:51 smithi012 ceph-mon[28184]: pgmap v160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:53 smithi154 ceph-mon[32578]: pgmap v161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:53 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:53 smithi077 ceph-mon[32280]: pgmap v161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:53 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:53.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:53 smithi012 ceph-mon[28184]: pgmap v161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:53.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:53 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:55.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:55 smithi154 ceph-mon[32578]: pgmap v162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:55.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:55 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:55.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:55 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:55.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:55 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:07:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:55 smithi077 ceph-mon[32280]: pgmap v162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:55 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:55 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:55 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:07:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:55 smithi012 ceph-mon[28184]: pgmap v162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:55 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:55 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:55 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:07:57.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:57 smithi154 ceph-mon[32578]: pgmap v163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:57 smithi077 ceph-mon[32280]: pgmap v163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:57 smithi012 ceph-mon[28184]: pgmap v163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:07:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:07:59 smithi154 ceph-mon[32578]: pgmap v164: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:07:59 smithi077 ceph-mon[32280]: pgmap v164: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:07:59 smithi012 ceph-mon[28184]: pgmap v164: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:01.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:01 smithi154 ceph-mon[32578]: pgmap v165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:01 smithi077 ceph-mon[32280]: pgmap v165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:01 smithi012 ceph-mon[28184]: pgmap v165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:03 smithi154 ceph-mon[32578]: pgmap v166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:03 smithi077 ceph-mon[32280]: pgmap v166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:03 smithi012 ceph-mon[28184]: pgmap v166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:05 smithi154 ceph-mon[32578]: pgmap v167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:05.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:05 smithi077 ceph-mon[32280]: pgmap v167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:05 smithi012 ceph-mon[28184]: pgmap v167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:07 smithi154 ceph-mon[32578]: pgmap v168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:07.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:07 smithi077 ceph-mon[32280]: pgmap v168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:07 smithi012 ceph-mon[28184]: pgmap v168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:09 smithi154 ceph-mon[32578]: pgmap v169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:09 smithi077 ceph-mon[32280]: pgmap v169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:09 smithi012 ceph-mon[28184]: pgmap v169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:11 smithi154 ceph-mon[32578]: pgmap v170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:11 smithi077 ceph-mon[32280]: pgmap v170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:11 smithi012 ceph-mon[28184]: pgmap v170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T22:08:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:13 smithi154 ceph-mon[32578]: pgmap v171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:13 smithi077 ceph-mon[32280]: pgmap v171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:13 smithi012 ceph-mon[28184]: pgmap v171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T22:08:15.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:15 smithi154 ceph-mon[32578]: pgmap v172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:15 smithi077 ceph-mon[32280]: pgmap v172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:15 smithi012 ceph-mon[28184]: pgmap v172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:17 smithi077 ceph-mon[32280]: pgmap v173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:17 smithi012 ceph-mon[28184]: pgmap v173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:17 smithi154 ceph-mon[32578]: pgmap v173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:08:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:19 smithi077 ceph-mon[32280]: pgmap v174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:08:19.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:19 smithi012 ceph-mon[28184]: pgmap v174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:08:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:19 smithi154 ceph-mon[32578]: pgmap v174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:08:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:21 smithi077 ceph-mon[32280]: pgmap v175: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:08:21.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:21 smithi012 ceph-mon[28184]: pgmap v175: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:08:21.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:21 smithi154 ceph-mon[32578]: pgmap v175: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T22:08:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:23 smithi077 ceph-mon[32280]: pgmap v176: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:08:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:23 smithi012 ceph-mon[28184]: pgmap v176: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:08:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:23 smithi154 ceph-mon[32578]: pgmap v176: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T22:08:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:25 smithi077 ceph-mon[32280]: pgmap v177: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:25 smithi012 ceph-mon[28184]: pgmap v177: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:25 smithi154 ceph-mon[32578]: pgmap v177: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:27.693 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:27 smithi154 ceph-mon[32578]: pgmap v178: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:27 smithi077 ceph-mon[32280]: pgmap v178: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:27.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:27 smithi012 ceph-mon[28184]: pgmap v178: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:29 smithi077 ceph-mon[32280]: pgmap v179: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:29 smithi012 ceph-mon[28184]: pgmap v179: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:29.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:29 smithi154 ceph-mon[32578]: pgmap v179: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:31.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:31 smithi077 ceph-mon[32280]: pgmap v180: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:31.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:31 smithi012 ceph-mon[28184]: pgmap v180: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:31 smithi154 ceph-mon[32578]: pgmap v180: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:33 smithi077 ceph-mon[32280]: pgmap v181: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:33 smithi012 ceph-mon[28184]: pgmap v181: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:33.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:33 smithi154 ceph-mon[32578]: pgmap v181: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:35.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:35 smithi077 ceph-mon[32280]: pgmap v182: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:35.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:35 smithi012 ceph-mon[28184]: pgmap v182: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:35.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:35 smithi154 ceph-mon[32578]: pgmap v182: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:37.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:37 smithi077 ceph-mon[32280]: pgmap v183: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:37.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:37 smithi012 ceph-mon[28184]: pgmap v183: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:37 smithi154 ceph-mon[32578]: pgmap v183: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:39 smithi077 ceph-mon[32280]: pgmap v184: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:39 smithi012 ceph-mon[28184]: pgmap v184: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:39.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:39 smithi154 ceph-mon[32578]: pgmap v184: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:41 smithi077 ceph-mon[32280]: pgmap v185: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:41.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:41 smithi012 ceph-mon[28184]: pgmap v185: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:41.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:41 smithi154 ceph-mon[32578]: pgmap v185: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:43 smithi077 ceph-mon[32280]: pgmap v186: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:43.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:43 smithi012 ceph-mon[28184]: pgmap v186: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:43.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:43 smithi154 ceph-mon[32578]: pgmap v186: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:45 smithi077 ceph-mon[32280]: pgmap v187: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:45.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:45 smithi012 ceph-mon[28184]: pgmap v187: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:45.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:45 smithi154 ceph-mon[32578]: pgmap v187: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:47.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:47 smithi077 ceph-mon[32280]: pgmap v188: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:47 smithi012 ceph-mon[28184]: pgmap v188: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:47.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:47 smithi154 ceph-mon[32578]: pgmap v188: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:49 smithi077 ceph-mon[32280]: pgmap v189: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:49.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:49 smithi012 ceph-mon[28184]: pgmap v189: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:49.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:49 smithi154 ceph-mon[32578]: pgmap v189: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:51 smithi012 ceph-mon[28184]: pgmap v190: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:51.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:51 smithi154 ceph-mon[32578]: pgmap v190: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:51.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:51 smithi077 ceph-mon[32280]: pgmap v190: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:53.803 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:53 smithi077 ceph-mon[32280]: pgmap v191: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:53.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:53 smithi012 ceph-mon[28184]: pgmap v191: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:53.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:53 smithi154 ceph-mon[32578]: pgmap v191: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:55 smithi012 ceph-mon[28184]: pgmap v192: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:55 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:55 smithi154 ceph-mon[32578]: pgmap v192: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:55.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:55 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:55 smithi077 ceph-mon[32280]: pgmap v192: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:55 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:56.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:56 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:56.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:56 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:56.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:56 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:08:56.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:56 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:56.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:56 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:56.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:56 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:08:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:56 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:56 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:56.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:56 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:08:57.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:57 smithi154 ceph-mon[32578]: pgmap v193: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:57.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:57 smithi012 ceph-mon[28184]: pgmap v193: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:57 smithi077 ceph-mon[32280]: pgmap v193: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:08:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:08:59 smithi012 ceph-mon[28184]: pgmap v194: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:59.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:08:59 smithi154 ceph-mon[32578]: pgmap v194: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:59.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:08:59 smithi077 ceph-mon[32280]: pgmap v194: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:01 smithi012 ceph-mon[28184]: pgmap v195: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:01 smithi154 ceph-mon[32578]: pgmap v195: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:01 smithi077 ceph-mon[32280]: pgmap v195: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:03 smithi012 ceph-mon[28184]: pgmap v196: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:03 smithi154 ceph-mon[32578]: pgmap v196: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:03.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:03 smithi077 ceph-mon[32280]: pgmap v196: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:05 smithi012 ceph-mon[28184]: pgmap v197: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:05.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:05 smithi154 ceph-mon[32578]: pgmap v197: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:05.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:05 smithi077 ceph-mon[32280]: pgmap v197: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:07 smithi012 ceph-mon[28184]: pgmap v198: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:07.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:07 smithi154 ceph-mon[32578]: pgmap v198: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:07.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:07 smithi077 ceph-mon[32280]: pgmap v198: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T22:09:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:09 smithi012 ceph-mon[28184]: pgmap v199: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:09.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:09 smithi012 ceph-mon[28184]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T22:09:09.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:09 smithi154 ceph-mon[32578]: pgmap v199: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:09.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:09 smithi154 ceph-mon[32578]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T22:09:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:09 smithi077 ceph-mon[32280]: pgmap v199: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T22:09:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:09 smithi077 ceph-mon[32280]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T22:09:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:11 smithi012 ceph-mon[28184]: pgmap v201: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T22:09:11.893 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:11 smithi154 ceph-mon[32578]: pgmap v201: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T22:09:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:11 smithi077 ceph-mon[32280]: pgmap v201: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T22:09:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:13 smithi012 ceph-mon[28184]: pgmap v202: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:13.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:13 smithi154 ceph-mon[32578]: pgmap v202: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:13.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:13 smithi077 ceph-mon[32280]: pgmap v202: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:15 smithi012 ceph-mon[28184]: pgmap v203: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:15.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:15 smithi154 ceph-mon[32578]: pgmap v203: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:15 smithi077 ceph-mon[32280]: pgmap v203: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:17 smithi012 ceph-mon[28184]: pgmap v204: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:17 smithi012 ceph-mon[28184]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T22:09:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:17 smithi154 ceph-mon[32578]: pgmap v204: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:17 smithi154 ceph-mon[32578]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T22:09:17.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:17 smithi077 ceph-mon[32280]: pgmap v204: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:17.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:17 smithi077 ceph-mon[32280]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T22:09:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:19 smithi154 ceph-mon[32578]: pgmap v206: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-26T22:09:19.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:19 smithi077 ceph-mon[32280]: pgmap v206: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-26T22:09:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:19 smithi012 ceph-mon[28184]: pgmap v206: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-26T22:09:21.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:21 smithi154 ceph-mon[32578]: pgmap v207: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:21.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:21 smithi077 ceph-mon[32280]: pgmap v207: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:22.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:21 smithi012 ceph-mon[28184]: pgmap v207: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:09:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:23 smithi154 ceph-mon[32578]: pgmap v208: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:23 smithi077 ceph-mon[32280]: pgmap v208: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:24.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:23 smithi012 ceph-mon[28184]: pgmap v208: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:25 smithi154 ceph-mon[32578]: pgmap v209: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:25.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:25 smithi077 ceph-mon[32280]: pgmap v209: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:25 smithi012 ceph-mon[28184]: pgmap v209: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:27.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:27 smithi154 ceph-mon[32578]: pgmap v210: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:27 smithi077 ceph-mon[32280]: pgmap v210: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:27 smithi012 ceph-mon[28184]: pgmap v210: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:09:29.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:29 smithi154 ceph-mon[32578]: pgmap v211: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:09:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:29 smithi077 ceph-mon[32280]: pgmap v211: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:09:30.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:29 smithi012 ceph-mon[28184]: pgmap v211: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:09:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:31 smithi154 ceph-mon[32578]: pgmap v212: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:31 smithi077 ceph-mon[32280]: pgmap v212: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:32.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:31 smithi012 ceph-mon[28184]: pgmap v212: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:33.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:33 smithi154 ceph-mon[32578]: pgmap v213: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:33 smithi077 ceph-mon[32280]: pgmap v213: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:33 smithi012 ceph-mon[28184]: pgmap v213: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:35.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:35 smithi154 ceph-mon[32578]: pgmap v214: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:35.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:35 smithi077 ceph-mon[32280]: pgmap v214: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:36.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:35 smithi012 ceph-mon[28184]: pgmap v214: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:37 smithi154 ceph-mon[32578]: pgmap v215: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:37.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:37 smithi077 ceph-mon[32280]: pgmap v215: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:38.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:37 smithi012 ceph-mon[28184]: pgmap v215: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:09:39.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:39 smithi154 ceph-mon[32578]: pgmap v216: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:39.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:39 smithi077 ceph-mon[32280]: pgmap v216: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:40.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:39 smithi012 ceph-mon[28184]: pgmap v216: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:41.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:41 smithi154 ceph-mon[32578]: pgmap v217: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:41.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:41 smithi077 ceph-mon[32280]: pgmap v217: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:42.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:41 smithi012 ceph-mon[28184]: pgmap v217: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:43.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:43 smithi154 ceph-mon[32578]: pgmap v218: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:43 smithi077 ceph-mon[32280]: pgmap v218: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:44.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:43 smithi012 ceph-mon[28184]: pgmap v218: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:45 smithi077 ceph-mon[32280]: pgmap v219: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:46.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:45 smithi012 ceph-mon[28184]: pgmap v219: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:46.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:45 smithi154 ceph-mon[32578]: pgmap v219: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:47 smithi077 ceph-mon[32280]: pgmap v220: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:48.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:47 smithi012 ceph-mon[28184]: pgmap v220: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:48.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:47 smithi154 ceph-mon[32578]: pgmap v220: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:49.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:49 smithi077 ceph-mon[32280]: pgmap v221: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:50.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:49 smithi012 ceph-mon[28184]: pgmap v221: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:50.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:49 smithi154 ceph-mon[32578]: pgmap v221: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:51.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:51 smithi077 ceph-mon[32280]: pgmap v222: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:52.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:51 smithi012 ceph-mon[28184]: pgmap v222: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:52.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:51 smithi154 ceph-mon[32578]: pgmap v222: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:52.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:52 smithi077 ceph-mon[32280]: pgmap v223: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:53.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:52 smithi012 ceph-mon[28184]: pgmap v223: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:53.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:52 smithi154 ceph-mon[32578]: pgmap v223: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:55.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:55 smithi077 ceph-mon[32280]: pgmap v224: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:55 smithi012 ceph-mon[28184]: pgmap v224: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:55 smithi154 ceph-mon[32578]: pgmap v224: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:57.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:57 smithi077 ceph-mon[32280]: pgmap v225: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:57.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:57 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:57 smithi012 ceph-mon[28184]: pgmap v225: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:57 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:57.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:57 smithi154 ceph-mon[32578]: pgmap v225: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:09:57.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:57 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:58.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:58 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:58.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:58 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:58.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:58 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:09:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:58 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:58 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:58 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:09:58.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:58 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:58.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:58 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:58.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:58 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:09:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:09:59 smithi012 ceph-mon[28184]: pgmap v226: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:09:59 smithi154 ceph-mon[32578]: pgmap v226: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:09:59 smithi077 ceph-mon[32280]: pgmap v226: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:00 smithi012 ceph-mon[28184]: overall HEALTH_OK 2024-04-26T22:10:00.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:00 smithi154 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T22:10:00.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:00 smithi077 ceph-mon[32280]: overall HEALTH_OK 2024-04-26T22:10:01.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:01 smithi012 ceph-mon[28184]: pgmap v227: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:01.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:01 smithi154 ceph-mon[32578]: pgmap v227: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:01 smithi077 ceph-mon[32280]: pgmap v227: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:03.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:03 smithi012 ceph-mon[28184]: pgmap v228: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:03 smithi154 ceph-mon[32578]: pgmap v228: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:03 smithi077 ceph-mon[32280]: pgmap v228: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:05.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:05 smithi012 ceph-mon[28184]: pgmap v229: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:05 smithi154 ceph-mon[32578]: pgmap v229: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:05.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:05 smithi077 ceph-mon[32280]: pgmap v229: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:07.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:07 smithi012 ceph-mon[28184]: pgmap v230: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:07 smithi154 ceph-mon[32578]: pgmap v230: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:07.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:07 smithi077 ceph-mon[32280]: pgmap v230: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:10:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:09 smithi012 ceph-mon[28184]: pgmap v231: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:09.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:09 smithi012 ceph-mon[28184]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T22:10:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:09 smithi154 ceph-mon[32578]: pgmap v231: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:09.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:09 smithi154 ceph-mon[32578]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T22:10:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:09 smithi077 ceph-mon[32280]: pgmap v231: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:10:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:09 smithi077 ceph-mon[32280]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T22:10:11.535 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:11 smithi077 ceph-mon[32280]: pgmap v233: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:10:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:11 smithi012 ceph-mon[28184]: pgmap v233: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:10:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:11 smithi154 ceph-mon[32578]: pgmap v233: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:10:13.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:13 smithi012 ceph-mon[28184]: pgmap v234: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:13 smithi154 ceph-mon[32578]: pgmap v234: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:13 smithi077 ceph-mon[32280]: pgmap v234: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:15.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:15 smithi012 ceph-mon[28184]: pgmap v235: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:15.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:15 smithi154 ceph-mon[32578]: pgmap v235: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:15 smithi077 ceph-mon[32280]: pgmap v235: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:17.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:17 smithi012 ceph-mon[28184]: pgmap v236: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:17.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:17 smithi012 ceph-mon[28184]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T22:10:17.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:17 smithi154 ceph-mon[32578]: pgmap v236: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:17.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:17 smithi154 ceph-mon[32578]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T22:10:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:17 smithi077 ceph-mon[32280]: pgmap v236: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:17 smithi077 ceph-mon[32280]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T22:10:19.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:19 smithi012 ceph-mon[28184]: pgmap v238: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-26T22:10:19.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:19 smithi154 ceph-mon[32578]: pgmap v238: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-26T22:10:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:19 smithi077 ceph-mon[32280]: pgmap v238: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-04-26T22:10:21.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:21 smithi012 ceph-mon[28184]: pgmap v239: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:21 smithi154 ceph-mon[32578]: pgmap v239: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:21 smithi077 ceph-mon[32280]: pgmap v239: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:10:23.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:23 smithi012 ceph-mon[28184]: pgmap v240: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:23 smithi154 ceph-mon[32578]: pgmap v240: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:23 smithi077 ceph-mon[32280]: pgmap v240: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:25.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:25 smithi012 ceph-mon[28184]: pgmap v241: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:25 smithi154 ceph-mon[32578]: pgmap v241: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:25 smithi077 ceph-mon[32280]: pgmap v241: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:27.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:27 smithi154 ceph-mon[32578]: pgmap v242: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:27 smithi077 ceph-mon[32280]: pgmap v242: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:27.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:27 smithi012 ceph-mon[28184]: pgmap v242: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:10:29.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:29 smithi154 ceph-mon[32578]: pgmap v243: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:10:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:29 smithi077 ceph-mon[32280]: pgmap v243: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:10:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:29 smithi012 ceph-mon[28184]: pgmap v243: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:10:31.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:31 smithi154 ceph-mon[32578]: pgmap v244: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:31 smithi077 ceph-mon[32280]: pgmap v244: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:31.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:31 smithi012 ceph-mon[28184]: pgmap v244: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:33.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:33 smithi154 ceph-mon[32578]: pgmap v245: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:33 smithi077 ceph-mon[32280]: pgmap v245: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:33 smithi012 ceph-mon[28184]: pgmap v245: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:35.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:35 smithi154 ceph-mon[32578]: pgmap v246: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:35.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:35 smithi077 ceph-mon[32280]: pgmap v246: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:35.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:35 smithi012 ceph-mon[28184]: pgmap v246: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:37.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:37 smithi154 ceph-mon[32578]: pgmap v247: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:37.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:37 smithi077 ceph-mon[32280]: pgmap v247: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:37.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:37 smithi012 ceph-mon[28184]: pgmap v247: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:39.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:39 smithi154 ceph-mon[32578]: pgmap v248: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:39 smithi077 ceph-mon[32280]: pgmap v248: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:39 smithi012 ceph-mon[28184]: pgmap v248: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:41.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:41 smithi154 ceph-mon[32578]: pgmap v249: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:41 smithi077 ceph-mon[32280]: pgmap v249: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:41.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:41 smithi012 ceph-mon[28184]: pgmap v249: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:43.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:42 smithi154 ceph-mon[32578]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:43.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:42 smithi077 ceph-mon[32280]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:43.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:42 smithi012 ceph-mon[28184]: pgmap v250: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:45.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:45 smithi012 ceph-mon[28184]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:45.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:45 smithi154 ceph-mon[32578]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:45 smithi077 ceph-mon[32280]: pgmap v251: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:47.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:47 smithi012 ceph-mon[28184]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:47.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:47 smithi154 ceph-mon[32578]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:47.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:47 smithi077 ceph-mon[32280]: pgmap v252: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:49 smithi012 ceph-mon[28184]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:49 smithi154 ceph-mon[32578]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:49 smithi077 ceph-mon[32280]: pgmap v253: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:51.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:51 smithi012 ceph-mon[28184]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:51.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:51 smithi154 ceph-mon[32578]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:51 smithi077 ceph-mon[32280]: pgmap v254: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:53 smithi012 ceph-mon[28184]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:53 smithi154 ceph-mon[32578]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:53 smithi077 ceph-mon[32280]: pgmap v255: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:55 smithi012 ceph-mon[28184]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:55 smithi154 ceph-mon[32578]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:55 smithi077 ceph-mon[32280]: pgmap v256: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:57 smithi012 ceph-mon[28184]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:57.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:57 smithi154 ceph-mon[32578]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:57 smithi077 ceph-mon[32280]: pgmap v257: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:10:58.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:58 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:58.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:58 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:58.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:58 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:10:59 smithi012 ceph-mon[28184]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:59.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:10:59 smithi154 ceph-mon[32578]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:10:59 smithi077 ceph-mon[32280]: pgmap v258: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:00 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:00 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:00.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:00 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:11:00.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:00 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:00.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:00 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:00.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:00 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:11:00.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:00 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:00.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:00 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:00.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:00 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:11:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:01 smithi012 ceph-mon[28184]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:01 smithi154 ceph-mon[32578]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:01.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:01 smithi077 ceph-mon[32280]: pgmap v259: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:03 smithi012 ceph-mon[28184]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:03.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:03 smithi154 ceph-mon[32578]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:03.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:03 smithi077 ceph-mon[32280]: pgmap v260: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:05 smithi012 ceph-mon[28184]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:05.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:05 smithi154 ceph-mon[32578]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:05.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:05 smithi077 ceph-mon[32280]: pgmap v261: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:07 smithi012 ceph-mon[28184]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:07.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:07 smithi154 ceph-mon[32578]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:07.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:07 smithi077 ceph-mon[32280]: pgmap v262: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:11:09.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:09 smithi154 ceph-mon[32578]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:09.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:09 smithi154 ceph-mon[32578]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T22:11:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:09 smithi077 ceph-mon[32280]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:09.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:09 smithi077 ceph-mon[32280]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T22:11:10.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:09 smithi012 ceph-mon[28184]: pgmap v263: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:11:10.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:09 smithi012 ceph-mon[28184]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T22:11:11.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:11 smithi154 ceph-mon[32578]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:11:11.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:11 smithi077 ceph-mon[32280]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:11:12.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:11 smithi012 ceph-mon[28184]: pgmap v265: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:11:13.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:13 smithi154 ceph-mon[32578]: pgmap v266: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:13.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:13 smithi077 ceph-mon[32280]: pgmap v266: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:14.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:13 smithi012 ceph-mon[28184]: pgmap v266: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:15.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:15 smithi154 ceph-mon[32578]: pgmap v267: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:15.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:15 smithi077 ceph-mon[32280]: pgmap v267: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:16.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:15 smithi012 ceph-mon[28184]: pgmap v267: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:17.876 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:17 smithi012 ceph-mon[28184]: pgmap v268: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:17.877 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:17 smithi012 ceph-mon[28184]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T22:11:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:17 smithi154 ceph-mon[32578]: pgmap v268: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:17.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:17 smithi154 ceph-mon[32578]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T22:11:17.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:17 smithi077 ceph-mon[32280]: pgmap v268: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:17.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:17 smithi077 ceph-mon[32280]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T22:11:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:19 smithi154 ceph-mon[32578]: pgmap v270: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 212 B/s wr, 2 op/s 2024-04-26T22:11:19.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:19 smithi077 ceph-mon[32280]: pgmap v270: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 212 B/s wr, 2 op/s 2024-04-26T22:11:20.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:19 smithi012 ceph-mon[28184]: pgmap v270: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 212 B/s wr, 2 op/s 2024-04-26T22:11:21.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:21 smithi154 ceph-mon[32578]: pgmap v271: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:21.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:21 smithi077 ceph-mon[32280]: pgmap v271: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:22.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:21 smithi012 ceph-mon[28184]: pgmap v271: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:11:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:23 smithi154 ceph-mon[32578]: pgmap v272: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:23.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:23 smithi077 ceph-mon[32280]: pgmap v272: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:24.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:23 smithi012 ceph-mon[28184]: pgmap v272: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:25 smithi154 ceph-mon[32578]: pgmap v273: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:25.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:25 smithi077 ceph-mon[32280]: pgmap v273: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:26.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:25 smithi012 ceph-mon[28184]: pgmap v273: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:27.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:27 smithi154 ceph-mon[32578]: pgmap v274: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:27.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:27 smithi077 ceph-mon[32280]: pgmap v274: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:28.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:27 smithi012 ceph-mon[28184]: pgmap v274: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:11:29.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:29 smithi154 ceph-mon[32578]: pgmap v275: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:11:29.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:29 smithi077 ceph-mon[32280]: pgmap v275: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:11:30.012 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:29 smithi012 ceph-mon[28184]: pgmap v275: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:11:31.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:31 smithi077 ceph-mon[32280]: pgmap v276: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:32.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:31 smithi012 ceph-mon[28184]: pgmap v276: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:32.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:31 smithi154 ceph-mon[32578]: pgmap v276: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:33.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:33 smithi077 ceph-mon[32280]: pgmap v277: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:34.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:33 smithi012 ceph-mon[28184]: pgmap v277: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:34.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:33 smithi154 ceph-mon[32578]: pgmap v277: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:35.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:35 smithi077 ceph-mon[32280]: pgmap v278: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:36.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:35 smithi012 ceph-mon[28184]: pgmap v278: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:36.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:35 smithi154 ceph-mon[32578]: pgmap v278: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:37.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:37 smithi077 ceph-mon[32280]: pgmap v279: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:38.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:37 smithi012 ceph-mon[28184]: pgmap v279: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:38.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:37 smithi154 ceph-mon[32578]: pgmap v279: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:39.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:39 smithi077 ceph-mon[32280]: pgmap v280: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:40.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:39 smithi012 ceph-mon[28184]: pgmap v280: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:40.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:39 smithi154 ceph-mon[32578]: pgmap v280: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:41.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:41 smithi077 ceph-mon[32280]: pgmap v281: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:42.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:41 smithi012 ceph-mon[28184]: pgmap v281: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:42.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:41 smithi154 ceph-mon[32578]: pgmap v281: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:43.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:43 smithi077 ceph-mon[32280]: pgmap v282: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:44.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:43 smithi012 ceph-mon[28184]: pgmap v282: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:44.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:43 smithi154 ceph-mon[32578]: pgmap v282: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:45.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:45 smithi077 ceph-mon[32280]: pgmap v283: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:46.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:45 smithi012 ceph-mon[28184]: pgmap v283: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:46.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:45 smithi154 ceph-mon[32578]: pgmap v283: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:46.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:46 smithi077 ceph-mon[32280]: pgmap v284: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:47.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:46 smithi012 ceph-mon[28184]: pgmap v284: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:47.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:46 smithi154 ceph-mon[32578]: pgmap v284: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:49.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:49 smithi012 ceph-mon[28184]: pgmap v285: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:49.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:49 smithi154 ceph-mon[32578]: pgmap v285: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:49.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:49 smithi077 ceph-mon[32280]: pgmap v285: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:51.575 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:51 smithi012 ceph-mon[28184]: pgmap v286: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:51.643 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:51 smithi154 ceph-mon[32578]: pgmap v286: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:51.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:51 smithi077 ceph-mon[32280]: pgmap v286: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:53 smithi012 ceph-mon[28184]: pgmap v287: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:53.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:53 smithi154 ceph-mon[32578]: pgmap v287: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:53.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:53 smithi077 ceph-mon[32280]: pgmap v287: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:55.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:55 smithi012 ceph-mon[28184]: pgmap v288: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:55.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:55 smithi154 ceph-mon[32578]: pgmap v288: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:55.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:55 smithi077 ceph-mon[32280]: pgmap v288: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:57.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:57 smithi012 ceph-mon[28184]: pgmap v289: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:57.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:57 smithi154 ceph-mon[32578]: pgmap v289: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:57.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:57 smithi077 ceph-mon[32280]: pgmap v289: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:11:59.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:11:59 smithi012 ceph-mon[28184]: pgmap v290: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:59.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:11:59 smithi154 ceph-mon[32578]: pgmap v290: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:59.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:11:59 smithi077 ceph-mon[32280]: pgmap v290: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:00.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:00 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:00.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:00 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:00.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:00 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:01 smithi154 ceph-mon[32578]: pgmap v291: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:01.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:01 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:01.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:01 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:01 smithi077 ceph-mon[32280]: pgmap v291: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:01 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:01 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:01 smithi012 ceph-mon[28184]: pgmap v291: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:01 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:01 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:02.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:02 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:12:02.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:02 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:12:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:02 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:12:03.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:03 smithi154 ceph-mon[32578]: pgmap v292: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:03.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:03 smithi077 ceph-mon[32280]: pgmap v292: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:03.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:03 smithi012 ceph-mon[28184]: pgmap v292: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:05.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:05 smithi154 ceph-mon[32578]: pgmap v293: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:05.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:05 smithi077 ceph-mon[32280]: pgmap v293: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:05.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:05 smithi012 ceph-mon[28184]: pgmap v293: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:07.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:07 smithi154 ceph-mon[32578]: pgmap v294: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:07.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:07 smithi077 ceph-mon[32280]: pgmap v294: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:07.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:07 smithi012 ceph-mon[28184]: pgmap v294: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:12:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:09 smithi154 ceph-mon[32578]: pgmap v295: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:09.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:09 smithi154 ceph-mon[32578]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T22:12:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:09 smithi077 ceph-mon[32280]: pgmap v295: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:09.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:09 smithi077 ceph-mon[32280]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T22:12:09.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:09 smithi012 ceph-mon[28184]: pgmap v295: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:12:09.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:09 smithi012 ceph-mon[28184]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T22:12:11.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:11 smithi154 ceph-mon[32578]: pgmap v297: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:12:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:11 smithi077 ceph-mon[32280]: pgmap v297: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:12:11.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:11 smithi012 ceph-mon[28184]: pgmap v297: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:12:13.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:13 smithi154 ceph-mon[32578]: pgmap v298: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:13.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:13 smithi077 ceph-mon[32280]: pgmap v298: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:13 smithi012 ceph-mon[28184]: pgmap v298: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:15.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:15 smithi154 ceph-mon[32578]: pgmap v299: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:15 smithi077 ceph-mon[32280]: pgmap v299: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:15 smithi012 ceph-mon[28184]: pgmap v299: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:17 smithi077 ceph-mon[32280]: pgmap v300: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:17 smithi077 ceph-mon[32280]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T22:12:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:17 smithi012 ceph-mon[28184]: pgmap v300: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:17.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:17 smithi012 ceph-mon[28184]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T22:12:17.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:17 smithi154 ceph-mon[32578]: pgmap v300: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:17.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:17 smithi154 ceph-mon[32578]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T22:12:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:19 smithi077 ceph-mon[32280]: pgmap v302: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-04-26T22:12:19.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:19 smithi012 ceph-mon[28184]: pgmap v302: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-04-26T22:12:19.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:19 smithi154 ceph-mon[32578]: pgmap v302: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-04-26T22:12:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:21 smithi077 ceph-mon[32280]: pgmap v303: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:21.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:21 smithi012 ceph-mon[28184]: pgmap v303: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:21.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:21 smithi154 ceph-mon[32578]: pgmap v303: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:12:23.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:23 smithi077 ceph-mon[32280]: pgmap v304: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:23.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:23 smithi012 ceph-mon[28184]: pgmap v304: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:23.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:23 smithi154 ceph-mon[32578]: pgmap v304: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:25 smithi077 ceph-mon[32280]: pgmap v305: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:25.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:25 smithi012 ceph-mon[28184]: pgmap v305: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:25.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:25 smithi154 ceph-mon[32578]: pgmap v305: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:27.693 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:27 smithi154 ceph-mon[32578]: pgmap v306: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:27 smithi077 ceph-mon[32280]: pgmap v306: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:27.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:27 smithi012 ceph-mon[28184]: pgmap v306: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:12:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:29 smithi077 ceph-mon[32280]: pgmap v307: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:12:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:29 smithi012 ceph-mon[28184]: pgmap v307: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:12:29.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:29 smithi154 ceph-mon[32578]: pgmap v307: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 90 B/s wr, 2 op/s 2024-04-26T22:12:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:31 smithi077 ceph-mon[32280]: pgmap v308: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:31.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:31 smithi012 ceph-mon[28184]: pgmap v308: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:31.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:31 smithi154 ceph-mon[32578]: pgmap v308: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:33.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:33 smithi077 ceph-mon[32280]: pgmap v309: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:33 smithi012 ceph-mon[28184]: pgmap v309: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:33.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:33 smithi154 ceph-mon[32578]: pgmap v309: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:35.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:35 smithi077 ceph-mon[32280]: pgmap v310: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:35.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:35 smithi012 ceph-mon[28184]: pgmap v310: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:35.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:35 smithi154 ceph-mon[32578]: pgmap v310: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:37.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:37 smithi077 ceph-mon[32280]: pgmap v311: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:37.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:37 smithi012 ceph-mon[28184]: pgmap v311: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:37.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:37 smithi154 ceph-mon[32578]: pgmap v311: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:38.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T22:12:38.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 -- 172.21.15.12:0/352852141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 msgr2=0x7f86081119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 --2- 172.21.15.12:0/352852141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f86081119c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8600009ca0 tx=0x7f860002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 -- 172.21.15.12:0/352852141 shutdown_connections 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 --2- 172.21.15.12:0/352852141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f86081119c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 --2- 172.21.15.12:0/352852141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8608104b40 0x7f8608104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 --2- 172.21.15.12:0/352852141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f860810ac00 0x7f860810afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 -- 172.21.15.12:0/352852141 >> 172.21.15.12:0/352852141 conn(0x7f8608100630 msgr2=0x7f8608102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 -- 172.21.15.12:0/352852141 shutdown_connections 2024-04-26T22:12:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.687+0000 7f860ea00640 1 -- 172.21.15.12:0/352852141 wait complete. 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 Processor -- start 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 -- start start 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f860810ac00 0x7f860819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8608114e40 con 0x7f8608105500 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8608114cc0 con 0x7f860810ac00 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.688+0000 7f860ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8608114fc0 con 0x7f8608104b40 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:38.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f8607e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f8607e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:48806/0 (socket says 172.21.15.12:48806) 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f8607e00640 1 -- 172.21.15.12:0/2526088567 learned_addr learned my addr 172.21.15.12:0/2526088567 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43356/0 (socket says 172.21.15.12:43356) 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f860ca00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f860810ac00 0x7f860819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 -- 172.21.15.12:0/2526088567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 msgr2=0x7f860819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 -- 172.21.15.12:0/2526088567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f860810ac00 msgr2=0x7f860819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f860810ac00 0x7f860819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f85ff400640 1 -- 172.21.15.12:0/2526088567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a01f0 con 0x7f8608105500 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.689+0000 7f8607e00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f85ff400640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9f0 tx=0x7f85f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000ccc0 con 0x7f8608105500 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a04e0 con 0x7f8608105500 2024-04-26T22:12:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f00108e0 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0018820 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.690+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a7d70 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.691+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86080506c0 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.693+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f85f0010430 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.695+0000 7f8605600640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 0x7f85ec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.695+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7f85f00d0ef0 con 0x7f8608105500 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.695+0000 7f8607e00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 0x7f85ec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.696+0000 7f8607e00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 0x7f85ec0a6280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011500 tx=0x7f85f8005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.700+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f85f009cae0 con 0x7f8608105500 2024-04-26T22:12:38.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.934+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f86081022c0 con 0x7f85ec0a3dc0 2024-04-26T22:12:38.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.947+0000 7f8605600640 1 -- 172.21.15.12:0/2526088567 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f86081022c0 con 0x7f85ec0a3dc0 2024-04-26T22:12:38.949 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T22:02:37", "first": "2024-04-26T22:03:00", "last": "2024-04-26T22:12:00", "last_pruned": "2024-04-26T22:12:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-26T22:12:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.950+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 msgr2=0x7f85ec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.950+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 0x7f85ec0a6280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011500 tx=0x7f85f8005f90 comp rx=0 tx=0).stop 2024-04-26T22:12:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.950+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 msgr2=0x7f860819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:38.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.950+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9f0 tx=0x7f85f000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.950+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 shutdown_connections 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f85ec0a3dc0 0x7f85ec0a6280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f860810ac00 0x7f860819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8608105500 0x7f860819b740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 --2- 172.21.15.12:0/2526088567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8608104b40 0x7f860819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 >> 172.21.15.12:0/2526088567 conn(0x7f8608100630 msgr2=0x7f8608101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 shutdown_connections 2024-04-26T22:12:38.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:38.951+0000 7f860ea00640 1 -- 172.21.15.12:0/2526088567 wait complete. 2024-04-26T22:12:38.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T22:12:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3724396346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74075620 msgr2=0x7f5c7410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3724396346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74075620 0x7f5c7410eaa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009900 tx=0x7f5c6002a480 comp rx=0 tx=0).stop 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3724396346 shutdown_connections 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3724396346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7410efe0 0x7f5c741113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3724396346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c74075620 0x7f5c7410eaa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3724396346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c74074d00 0x7f5c740750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3724396346 >> 172.21.15.12:0/3724396346 conn(0x7f5c740fe270 msgr2=0x7f5c74100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3724396346 shutdown_connections 2024-04-26T22:12:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.175+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3724396346 wait complete. 2024-04-26T22:12:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.176+0000 7f5c7ae00640 1 Processor -- start 2024-04-26T22:12:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.176+0000 7f5c7ae00640 1 -- start start 2024-04-26T22:12:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 0x7f5c7419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 0x7f5c7419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 0x7f5c741a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74114c40 con 0x7f5c7410efe0 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74114ac0 con 0x7f5c7419fc70 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74114dc0 con 0x7f5c74074d00 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c79000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 0x7f5c7419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c79a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 0x7f5c7419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c79000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 0x7f5c7419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36300/0 (socket says 172.21.15.12:36300) 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c79000640 1 -- 172.21.15.12:0/3151867159 learned_addr learned my addr 172.21.15.12:0/3151867159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.177+0000 7f5c7a400640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 0x7f5c741a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 -- 172.21.15.12:0/3151867159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 msgr2=0x7f5c741a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 0x7f5c741a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 -- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 msgr2=0x7f5c7419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 0x7f5c7419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 -- 172.21.15.12:0/3151867159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c741a0760 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c7a400640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 0x7f5c741a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c79a00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 0x7f5c7419b3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e9f0 tx=0x7f5c6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cd80 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c680035f0 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68010670 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c741a0a50 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.178+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741a81c0 con 0x7f5c74074d00 2024-04-26T22:12:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.180+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74050f10 con 0x7f5c74074d00 2024-04-26T22:12:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.182+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5c680037a0 con 0x7f5c74074d00 2024-04-26T22:12:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.183+0000 7f5c72000640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 0x7f5c5c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.183+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7f5c680d0ea0 con 0x7f5c74074d00 2024-04-26T22:12:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.184+0000 7f5c79000640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 0x7f5c5c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.184+0000 7f5c79000640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 0x7f5c5c0a6250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c60031440 tx=0x7f5c60005a40 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.186+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5c6809d0e0 con 0x7f5c74074d00 2024-04-26T22:12:39.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.423+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 --> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5c7402db40 con 0x7f5c5c0a3d90 2024-04-26T22:12:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.438+0000 7f5c72000640 1 -- 172.21.15.12:0/3151867159 <== mgr.25318 v2:172.21.15.12:6800/2662094264 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5c7402db40 con 0x7f5c5c0a3d90 2024-04-26T22:12:39.439 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.441+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 msgr2=0x7f5c5c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.441+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 0x7f5c5c0a6250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c60031440 tx=0x7f5c60005a40 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.441+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 msgr2=0x7f5c7419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.441+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 0x7f5c7419b3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e9f0 tx=0x7f5c6800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 shutdown_connections 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5c5c0a3d90 0x7f5c5c0a6250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7419fc70 0x7f5c741a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c7410efe0 0x7f5c7419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 --2- 172.21.15.12:0/3151867159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c74074d00 0x7f5c7419b3b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 >> 172.21.15.12:0/3151867159 conn(0x7f5c740fe270 msgr2=0x7f5c740ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 shutdown_connections 2024-04-26T22:12:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.442+0000 7f5c7ae00640 1 -- 172.21.15.12:0/3151867159 wait complete. 2024-04-26T22:12:39.463 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T22:12:39.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 -- 172.21.15.12:0/2262969397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f165410bc80 msgr2=0x7f165410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 --2- 172.21.15.12:0/2262969397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f165410bc80 0x7f165410e090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f164c009be0 tx=0x7f164c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 -- 172.21.15.12:0/2262969397 shutdown_connections 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 --2- 172.21.15.12:0/2262969397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f165410bc80 0x7f165410e090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 --2- 172.21.15.12:0/2262969397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16541031c0 0x7f165410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 --2- 172.21.15.12:0/2262969397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1654102810 0x7f1654102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 -- 172.21.15.12:0/2262969397 >> 172.21.15.12:0/2262969397 conn(0x7f1654077800 msgr2=0x7f1654077c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 -- 172.21.15.12:0/2262969397 shutdown_connections 2024-04-26T22:12:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.691+0000 7f165aa00640 1 -- 172.21.15.12:0/2262969397 wait complete. 2024-04-26T22:12:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.692+0000 7f165aa00640 1 Processor -- start 2024-04-26T22:12:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.692+0000 7f165aa00640 1 -- start start 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f165419f920 0x7f165419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16541129e0 con 0x7f165419f920 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1654112860 con 0x7f1654102810 2024-04-26T22:12:39.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165aa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1654112b60 con 0x7f165410bc80 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f165a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f165419f920 0x7f165419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f1658c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f1659600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f1658c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47484/0 (socket says 172.21.15.12:47484) 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f1659600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:48044/0 (socket says 172.21.15.12:48044) 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.693+0000 7f1658c00640 1 -- 172.21.15.12:0/7742177 learned_addr learned my addr 172.21.15.12:0/7742177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 -- 172.21.15.12:0/7742177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 msgr2=0x7f165419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 -- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f165419f920 msgr2=0x7f165419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f165419f920 0x7f165419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 -- 172.21.15.12:0/7742177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16541a0410 con 0x7f165410bc80 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1659600640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.694+0000 7f1658c00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164000e9f0 tx=0x7f164000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.695+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164000ccc0 con 0x7f165410bc80 2024-04-26T22:12:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.695+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16541a0700 con 0x7f165410bc80 2024-04-26T22:12:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.695+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16400054d0 con 0x7f165410bc80 2024-04-26T22:12:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.695+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640010430 con 0x7f165410bc80 2024-04-26T22:12:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.695+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541a7ea0 con 0x7f165410bc80 2024-04-26T22:12:39.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.696+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1654100c00 con 0x7f165410bc80 2024-04-26T22:12:39.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.698+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f16400046c0 con 0x7f165410bc80 2024-04-26T22:12:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.700+0000 7f1652000640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 0x7f163c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.700+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7f16400d0fc0 con 0x7f165410bc80 2024-04-26T22:12:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.700+0000 7f1659600640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 0x7f163c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.701+0000 7f1659600640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 0x7f163c0a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16480096c0 tx=0x7f1648009260 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.702+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f164009d200 con 0x7f165410bc80 2024-04-26T22:12:39.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:39 smithi077 ceph-mon[32280]: pgmap v312: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:39 smithi012 ceph-mon[28184]: pgmap v312: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:39.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:39 smithi154 ceph-mon[32578]: pgmap v312: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:39.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.913+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f165419c3b0 con 0x7f165410bc80 2024-04-26T22:12:39.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.915+0000 7f1652000640 1 -- 172.21.15.12:0/7742177 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25344 (secure 0 0 0) 0x7f164009cb00 con 0x7f165410bc80 2024-04-26T22:12:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:12:39.917 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":103,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:12:16.860049+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1038496765":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/3660687894":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/36130226":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/53237046":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/505816482":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2706265351":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/3660687894":"2024-04-27T22:02:35.907627+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:12:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 msgr2=0x7f163c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 0x7f163c0a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16480096c0 tx=0x7f1648009260 comp rx=0 tx=0).stop 2024-04-26T22:12:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 msgr2=0x7f165419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:39.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164000e9f0 tx=0x7f164000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 shutdown_connections 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f163c0a3d20 0x7f163c0a61e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f165419f920 0x7f165419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165410bc80 0x7f165419b5a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 --2- 172.21.15.12:0/7742177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102810 0x7f165419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 >> 172.21.15.12:0/7742177 conn(0x7f1654077800 msgr2=0x7f165410ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.918+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 shutdown_connections 2024-04-26T22:12:39.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:39.919+0000 7f165aa00640 1 -- 172.21.15.12:0/7742177 wait complete. 2024-04-26T22:12:39.940 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:12:39.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:12:40.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.164+0000 7f16d3200640 1 -- 172.21.15.12:0/1228641299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41027d0 msgr2=0x7f16d4102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.164+0000 7f16d3200640 1 --2- 172.21.15.12:0/1228641299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41027d0 0x7f16d4102c50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009930 tx=0x7f16c002a420 comp rx=0 tx=0).stop 2024-04-26T22:12:40.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 -- 172.21.15.12:0/1228641299 shutdown_connections 2024-04-26T22:12:40.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 --2- 172.21.15.12:0/1228641299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d4103190 0x7f16d410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 --2- 172.21.15.12:0/1228641299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41027d0 0x7f16d4102c50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 --2- 172.21.15.12:0/1228641299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d41087e0 0x7f16d4108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 -- 172.21.15.12:0/1228641299 >> 172.21.15.12:0/1228641299 conn(0x7f16d40fe300 msgr2=0x7f16d4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:40.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 -- 172.21.15.12:0/1228641299 shutdown_connections 2024-04-26T22:12:40.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.165+0000 7f16d3200640 1 -- 172.21.15.12:0/1228641299 wait complete. 2024-04-26T22:12:40.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 Processor -- start 2024-04-26T22:12:40.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 -- start start 2024-04-26T22:12:40.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 0x7f16d4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 0x7f16d41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4112af0 con 0x7f16d41027d0 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4112970 con 0x7f16d4103190 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.166+0000 7f16d3200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4112c70 con 0x7f16d41087e0 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47490/0 (socket says 172.21.15.12:47490) 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d2800640 1 -- 172.21.15.12:0/3849517470 learned_addr learned my addr 172.21.15.12:0/3849517470 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 0x7f16d41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1e00640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 0x7f16d4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 -- 172.21.15.12:0/3849517470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 msgr2=0x7f16d419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 -- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 msgr2=0x7f16d4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 0x7f16d4199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1400640 1 -- 172.21.15.12:0/3849517470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d419e010 con 0x7f16d4103190 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d1e00640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 0x7f16d4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.167+0000 7f16d2800640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16d1400640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 0x7f16d41995b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f16c002a940 tx=0x7f16c0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c003f070 con 0x7f16d4103190 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d419e2a0 con 0x7f16d4103190 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c0031b60 con 0x7f16d4103190 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0035030 con 0x7f16d4103190 2024-04-26T22:12:40.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.168+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d41a5b90 con 0x7f16d4103190 2024-04-26T22:12:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.169+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d4069670 con 0x7f16d4103190 2024-04-26T22:12:40.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.171+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f16c0042070 con 0x7f16d4103190 2024-04-26T22:12:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.173+0000 7f16caa00640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 0x7f16b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.173+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7f16c00f1540 con 0x7f16d4103190 2024-04-26T22:12:40.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.173+0000 7f16d1e00640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 0x7f16b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.174+0000 7f16d1e00640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 0x7f16b80a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005fd0 tx=0x7f16c4009500 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:40.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.175+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16c00bd130 con 0x7f16d4103190 2024-04-26T22:12:40.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.385+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16d410ad40 con 0x7f16d4103190 2024-04-26T22:12:40.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.386+0000 7f16caa00640 1 -- 172.21.15.12:0/3849517470 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f16c00c1430 con 0x7f16d4103190 2024-04-26T22:12:40.387 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.388+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 msgr2=0x7f16b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.388+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 0x7f16b80a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005fd0 tx=0x7f16c4009500 comp rx=0 tx=0).stop 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 msgr2=0x7f16d41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 0x7f16d41995b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f16c002a940 tx=0x7f16c0003c10 comp rx=0 tx=0).stop 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 shutdown_connections 2024-04-26T22:12:40.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f16b80a3d20 0x7f16b80a61e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16d41087e0 0x7f16d419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16d4103190 0x7f16d41995b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 --2- 172.21.15.12:0/3849517470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16d41027d0 0x7f16d4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 >> 172.21.15.12:0/3849517470 conn(0x7f16d40fe300 msgr2=0x7f16d40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 shutdown_connections 2024-04-26T22:12:40.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.389+0000 7f16d3200640 1 -- 172.21.15.12:0/3849517470 wait complete. 2024-04-26T22:12:40.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.625+0000 7fde2c400640 1 -- 172.21.15.12:0/2169410565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24108910 msgr2=0x7fde24108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.625+0000 7fde2c400640 1 --2- 172.21.15.12:0/2169410565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24108910 0x7fde24108cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fde18009930 tx=0x7fde1802a440 comp rx=0 tx=0).stop 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 -- 172.21.15.12:0/2169410565 shutdown_connections 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 --2- 172.21.15.12:0/2169410565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde241032c0 0x7fde2410f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 --2- 172.21.15.12:0/2169410565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24102900 0x7fde24102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 --2- 172.21.15.12:0/2169410565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24108910 0x7fde24108cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 -- 172.21.15.12:0/2169410565 >> 172.21.15.12:0/2169410565 conn(0x7fde24079d80 msgr2=0x7fde2407a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:40.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 -- 172.21.15.12:0/2169410565 shutdown_connections 2024-04-26T22:12:40.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.626+0000 7fde2c400640 1 -- 172.21.15.12:0/2169410565 wait complete. 2024-04-26T22:12:40.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 Processor -- start 2024-04-26T22:12:40.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 -- start start 2024-04-26T22:12:40.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde241032c0 0x7fde24199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde24108910 0x7fde2419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fde24112bf0 con 0x7fde24102900 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde24112a70 con 0x7fde24108910 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.627+0000 7fde2c400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde24112d70 con 0x7fde241032c0 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36374/0 (socket says 172.21.15.12:36374) 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 -- 172.21.15.12:0/872956709 learned_addr learned my addr 172.21.15.12:0/872956709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29000640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde241032c0 0x7fde24199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde2a400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde24108910 0x7fde2419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 -- 172.21.15.12:0/872956709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde241032c0 msgr2=0x7fde24199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde241032c0 0x7fde24199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 -- 172.21.15.12:0/872956709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde24108910 msgr2=0x7fde2419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde24108910 0x7fde2419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.628+0000 7fde29a00640 1 -- 172.21.15.12:0/872956709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2419e1b0 con 0x7fde24102900 2024-04-26T22:12:40.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde29a00640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde18005ab0 tx=0x7fde18003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1803f070 con 0x7fde24102900 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2419e440 con 0x7fde24102900 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde18031b50 con 0x7fde24102900 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18035030 con 0x7fde24102900 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.629+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241a5d30 con 0x7fde24102900 2024-04-26T22:12:40.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.631+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde24069ba0 con 0x7fde24102900 2024-04-26T22:12:40.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.633+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fde18042070 con 0x7fde24102900 2024-04-26T22:12:40.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.634+0000 7fde22000640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:40.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.634+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7fde180f11f0 con 0x7fde24102900 2024-04-26T22:12:40.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.635+0000 7fde29000640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:40.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.635+0000 7fde29000640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 0x7fde0c0a6260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde241022d0 tx=0x7fde1000b410 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:40.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.638+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fde180bd430 con 0x7fde24102900 2024-04-26T22:12:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:40 smithi077 ceph-mon[32280]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:40 smithi077 ceph-mon[32280]: from='client.25711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:40 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/7742177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:12:40.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:40 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3849517470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:12:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:40 smithi012 ceph-mon[28184]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:40 smithi012 ceph-mon[28184]: from='client.25711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:40 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/7742177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:12:40.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:40 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3849517470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:12:40.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.847+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde2410ae70 con 0x7fde24102900 2024-04-26T22:12:40.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.848+0000 7fde22000640 1 -- 172.21.15.12:0/872956709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fde180bcd30 con 0x7fde24102900 2024-04-26T22:12:40.850 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 msgr2=0x7fde0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 0x7fde0c0a6260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde241022d0 tx=0x7fde1000b410 comp rx=0 tx=0).stop 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 msgr2=0x7fde241991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde18005ab0 tx=0x7fde18003b90 comp rx=0 tx=0).stop 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 shutdown_connections 2024-04-26T22:12:40.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde24108910 0x7fde2419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde241032c0 0x7fde24199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 --2- 172.21.15.12:0/872956709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde24102900 0x7fde241991c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 >> 172.21.15.12:0/872956709 conn(0x7fde24079d80 msgr2=0x7fde24100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 shutdown_connections 2024-04-26T22:12:40.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:40.851+0000 7fde2c400640 1 -- 172.21.15.12:0/872956709 wait complete. 2024-04-26T22:12:40.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:12:40.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:40 smithi154 ceph-mon[32578]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:40 smithi154 ceph-mon[32578]: from='client.25711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T22:12:40.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:40 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/7742177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:12:40.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:40 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3849517470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.085+0000 7fca69e00640 1 -- 172.21.15.12:0/1186021654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 msgr2=0x7fca64102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.085+0000 7fca69e00640 1 --2- 172.21.15.12:0/1186021654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64102b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca50009930 tx=0x7fca5002a420 comp rx=0 tx=0).stop 2024-04-26T22:12:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 -- 172.21.15.12:0/1186021654 shutdown_connections 2024-04-26T22:12:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 --2- 172.21.15.12:0/1186021654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca6410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 --2- 172.21.15.12:0/1186021654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64102b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 --2- 172.21.15.12:0/1186021654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 0x7fca64108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 -- 172.21.15.12:0/1186021654 >> 172.21.15.12:0/1186021654 conn(0x7fca640fe200 msgr2=0x7fca64100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 -- 172.21.15.12:0/1186021654 shutdown_connections 2024-04-26T22:12:41.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.086+0000 7fca69e00640 1 -- 172.21.15.12:0/1186021654 wait complete. 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 Processor -- start 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 -- start start 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca64075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 0x7fca640756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fca641129f0 con 0x7fca641026d0 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca64112870 con 0x7fca64103090 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.087+0000 7fca69e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca64112b70 con 0x7fca641086e0 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca63e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca64075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36400/0 (socket says 172.21.15.12:36400) 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 -- 172.21.15.12:0/1245163390 learned_addr learned my addr 172.21.15.12:0/1245163390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca69400640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 0x7fca640756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 -- 172.21.15.12:0/1245163390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 msgr2=0x7fca640756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 0x7fca640756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 -- 172.21.15.12:0/1245163390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 msgr2=0x7fca64075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca64075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca68a00640 1 -- 172.21.15.12:0/1245163390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a7450 con 0x7fca641026d0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.088+0000 7fca63e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca64075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca68a00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e9f0 tx=0x7fca5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5800cd80 con 0x7fca641026d0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580054d0 con 0x7fca641026d0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58010690 con 0x7fca641026d0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a7740 con 0x7fca641026d0 2024-04-26T22:12:41.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.089+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a7c60 con 0x7fca641026d0 2024-04-26T22:12:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.090+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca64069ac0 con 0x7fca641026d0 2024-04-26T22:12:41.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.092+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fca58004e90 con 0x7fca641026d0 2024-04-26T22:12:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.094+0000 7fca61600640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 0x7fca4c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.094+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7fca580d0f80 con 0x7fca641026d0 2024-04-26T22:12:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.094+0000 7fca63e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 0x7fca4c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.095+0000 7fca63e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 0x7fca4c0a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca50031440 tx=0x7fca50005a20 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:41.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.098+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fca5809d1c0 con 0x7fca641026d0 2024-04-26T22:12:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.306+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca6410ac60 con 0x7fca641026d0 2024-04-26T22:12:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.307+0000 7fca61600640 1 -- 172.21.15.12:0/1245163390 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fca5809cac0 con 0x7fca641026d0 2024-04-26T22:12:41.309 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:12:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 msgr2=0x7fca4c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 0x7fca4c0a6260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca50031440 tx=0x7fca50005a20 comp rx=0 tx=0).stop 2024-04-26T22:12:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 msgr2=0x7fca64074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e9f0 tx=0x7fca5800edf0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 shutdown_connections 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fca4c0a3da0 0x7fca4c0a6260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.310+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca641086e0 0x7fca640756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.311+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca64103090 0x7fca64075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.311+0000 7fca69e00640 1 --2- 172.21.15.12:0/1245163390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca641026d0 0x7fca64074c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.311+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 >> 172.21.15.12:0/1245163390 conn(0x7fca640fe200 msgr2=0x7fca640fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.311+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 shutdown_connections 2024-04-26T22:12:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.311+0000 7fca69e00640 1 -- 172.21.15.12:0/1245163390 wait complete. 2024-04-26T22:12:41.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.548+0000 7fc12ae00640 1 -- 172.21.15.12:0/961444010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124075680 msgr2=0x7fc12410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.548+0000 7fc12ae00640 1 --2- 172.21.15.12:0/961444010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124075680 0x7fc12410eab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc110009900 tx=0x7fc11002a480 comp rx=0 tx=0).stop 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 -- 172.21.15.12:0/961444010 shutdown_connections 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 --2- 172.21.15.12:0/961444010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc12410eff0 0x7fc124111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 --2- 172.21.15.12:0/961444010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124075680 0x7fc12410eab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 --2- 172.21.15.12:0/961444010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc124074d60 0x7fc124075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 -- 172.21.15.12:0/961444010 >> 172.21.15.12:0/961444010 conn(0x7fc1240fe300 msgr2=0x7fc124100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 -- 172.21.15.12:0/961444010 shutdown_connections 2024-04-26T22:12:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.549+0000 7fc12ae00640 1 -- 172.21.15.12:0/961444010 wait complete. 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 Processor -- start 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 -- start start 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124074d60 0x7fc12419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc12410eff0 0x7fc12419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc124114c90 con 0x7fc124075680 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc124114b10 con 0x7fc124074d60 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.550+0000 7fc12ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc124114e10 con 0x7fc12410eff0 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36430/0 (socket says 172.21.15.12:36430) 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 -- 172.21.15.12:0/794698310 learned_addr learned my addr 172.21.15.12:0/794698310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc12a400640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc12410eff0 0x7fc12419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129a00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124074d60 0x7fc12419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 -- 172.21.15.12:0/794698310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc12410eff0 msgr2=0x7fc12419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc12410eff0 0x7fc12419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 -- 172.21.15.12:0/794698310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124074d60 msgr2=0x7fc12419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124074d60 0x7fc12419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 -- 172.21.15.12:0/794698310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a01e0 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.551+0000 7fc129000640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc110005ab0 tx=0x7fc110003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.552+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11003f070 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.552+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a0470 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.552+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc110031b60 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.552+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110035030 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.552+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a7d60 con 0x7fc124075680 2024-04-26T22:12:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.553+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124069ac0 con 0x7fc124075680 2024-04-26T22:12:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.555+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc110042070 con 0x7fc124075680 2024-04-26T22:12:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.556+0000 7fc122000640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 0x7fc10c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.557+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7fc1100f1290 con 0x7fc124075680 2024-04-26T22:12:41.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.557+0000 7fc129a00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 0x7fc10c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:41.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.558+0000 7fc129a00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 0x7fc10c0a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc118009bd0 tx=0x7fc118005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:41.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.561+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc1100bce80 con 0x7fc124075680 2024-04-26T22:12:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:41 smithi077 ceph-mon[32280]: pgmap v313: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:41 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/872956709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:41 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1245163390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.765+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc12410a170 con 0x7fc124075680 2024-04-26T22:12:41.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.766+0000 7fc122000640 1 -- 172.21.15.12:0/794698310 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7fc1100c1180 con 0x7fc124075680 2024-04-26T22:12:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.768+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 msgr2=0x7fc10c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.768+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 0x7fc10c0a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc118009bd0 tx=0x7fc118005f90 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.768+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 msgr2=0x7fc12419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.768+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc110005ab0 tx=0x7fc110003c10 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 shutdown_connections 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc10c0a3d20 0x7fc10c0a61e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc12410eff0 0x7fc12419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc124075680 0x7fc12419b730 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 --2- 172.21.15.12:0/794698310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc124074d60 0x7fc12419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 >> 172.21.15.12:0/794698310 conn(0x7fc1240fe300 msgr2=0x7fc1240ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 shutdown_connections 2024-04-26T22:12:41.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.769+0000 7fc12ae00640 1 -- 172.21.15.12:0/794698310 wait complete. 2024-04-26T22:12:41.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:41 smithi012 ceph-mon[28184]: pgmap v313: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:41.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:41 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/872956709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:41 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1245163390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T22:12:41.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:41 smithi154 ceph-mon[32578]: pgmap v313: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:41 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/872956709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:41 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1245163390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 -- 172.21.15.12:0/3023781835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b0077780 msgr2=0x7fb6b0113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/3023781835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b0077780 0x7fb6b0113260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb698009890 tx=0x7fb69802a360 comp rx=0 tx=0).stop 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 -- 172.21.15.12:0/3023781835 shutdown_connections 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/3023781835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b01137a0 0x7fb6b0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/3023781835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b0077780 0x7fb6b0113260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/3023781835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0076e60 0x7fb6b0077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:41.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 -- 172.21.15.12:0/3023781835 >> 172.21.15.12:0/3023781835 conn(0x7fb6b0100630 msgr2=0x7fb6b0102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:41.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 -- 172.21.15.12:0/3023781835 shutdown_connections 2024-04-26T22:12:41.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.997+0000 7fb6b6a00640 1 -- 172.21.15.12:0/3023781835 wait complete. 2024-04-26T22:12:42.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 Processor -- start 2024-04-26T22:12:42.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 -- start start 2024-04-26T22:12:42.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b0076e60 0x7fb6b01a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:42.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:42.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 0x7fb6b019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01191c0 con 0x7fb6b0076e60 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0119040 con 0x7fb6b01137a0 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.998+0000 7fb6b6a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0119340 con 0x7fb6b0077780 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6afe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b0076e60 0x7fb6b01a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6b4a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 0x7fb6b019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:47554/0 (socket says 172.21.15.12:47554) 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 -- 172.21.15.12:0/2962183590 learned_addr learned my addr 172.21.15.12:0/2962183590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 -- 172.21.15.12:0/2962183590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 msgr2=0x7fb6b019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 0x7fb6b019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 -- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b0076e60 msgr2=0x7fb6b01a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b0076e60 0x7fb6b01a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6a7e00640 1 -- 172.21.15.12:0/2962183590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b019fdf0 con 0x7fb6b0077780 2024-04-26T22:12:42.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:41.999+0000 7fb6b4a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 0x7fb6b019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:12:42.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6a7e00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb698032d80 tx=0x7fb698032db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:42.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698037040 con 0x7fb6b0077780 2024-04-26T22:12:42.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b01a0080 con 0x7fb6b0077780 2024-04-26T22:12:42.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb698041070 con 0x7fb6b0077780 2024-04-26T22:12:42.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698003850 con 0x7fb6b0077780 2024-04-26T22:12:42.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.000+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01ac330 con 0x7fb6b0077780 2024-04-26T22:12:42.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.001+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b0069ba0 con 0x7fb6b0077780 2024-04-26T22:12:42.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.003+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb6980039f0 con 0x7fb6b0077780 2024-04-26T22:12:42.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.004+0000 7fb6ad600640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:12:42.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.005+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9547+0+0 (secure 0 0 0) 0x7fb6980f1060 con 0x7fb6b0077780 2024-04-26T22:12:42.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.005+0000 7fb6afe00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:12:42.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.005+0000 7fb6afe00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 0x7fb6940a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009bd0 tx=0x7fb6a0005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:12:42.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.007+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb6980308a0 con 0x7fb6b0077780 2024-04-26T22:12:42.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.259+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb6b010e930 con 0x7fb6b0077780 2024-04-26T22:12:42.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.272+0000 7fb6ad600640 1 -- 172.21.15.12:0/2962183590 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746060 (secure 0 0 0) 0x7fb6980328c0 con 0x7fb6b0077780 2024-04-26T22:12:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 25, 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 25318, 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2662094264 2024-04-26T22:12:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2662094264 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/2662094264", 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-04-26T22:02:35.908187+0000", 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-04-26T22:12:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 25310, 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-26T22:12:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-26T22:12:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-26T22:12:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-26T22:12:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-26T22:12:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:12:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:12:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:12:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-26T22:12:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:12:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:12:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:12:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:12:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:12:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-26T22:12:42.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-26T22:12:42.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-26T22:12:42.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.384 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-26T22:12:42.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.386 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.387 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.388 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.396 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-26T22:12:42.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-26T22:12:42.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.407 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-26T22:12:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-26T22:12:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-26T22:12:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-26T22:12:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:12:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:12:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:12:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:12:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:12:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:12:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:12:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-26T22:12:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.520 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.521 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.522 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-26T22:12:42.524 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.525 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.526 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.531 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.532 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.533 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.534 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.544 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.547 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.556 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.561 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.562 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.563 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.566 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.572 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.573 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.574 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.581 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.584 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.590 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.596 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-26T22:12:42.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:42.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.615 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.618 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.627 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.628 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.630 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.631 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.638 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.640 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-26T22:12:42.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.643 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-26T22:12:42.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-26T22:12:42.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-26T22:12:42.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-26T22:12:42.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-26T22:12:42.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-26T22:12:42.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-26T22:12:42.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-26T22:12:42.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-26T22:12:42.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:12:42.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.657 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.668 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.671 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.673 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.676 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-26T22:12:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-26T22:12:42.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.684 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.686 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.697 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.698 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-26T22:12:42.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.707 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.713 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-26T22:12:42.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-26T22:12:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.723 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 25315, 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-26T22:12:42.731 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:12:42.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.733 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-26T22:12:42.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-26T22:12:42.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-26T22:12:42.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-26T22:12:42.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-26T22:12:42.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-26T22:12:42.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-26T22:12:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.755 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.757 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-26T22:12:42.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:42.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-26T22:12:42.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:12:42.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:42.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.776 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.777 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.778 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-26T22:12:42.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-26T22:12:42.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-26T22:12:42.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-26T22:12:42.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.792 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-26T22:12:42.800 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.811 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:12:42.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:12:42.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-26T22:12:42.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-26T22:12:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-26T22:12:42.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:12:42.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:12:42.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.834 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.835 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:12:42.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:12:42.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:12:42.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:12:42.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:12:42.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.861 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:12:42.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:12:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:12:42.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:12:42.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:12:42.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:12:42.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:12:42.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.874 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.875 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.876 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.877 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-26T22:12:42.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-26T22:12:42.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:42.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.895 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.896 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.899 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-26T22:12:42.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:42.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-26T22:12:42.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.914 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.915 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:42.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-26T22:12:42.927 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.928 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.930 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.931 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-26T22:12:42.936 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:12:42.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.941 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.942 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.944 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:42.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.952 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.953 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.961 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:12:42.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:12:42.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.966 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-26T22:12:42.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.986 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-26T22:12:42.988 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.989 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:42.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:42.997 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:42.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:42.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.002 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.004 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.011 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.015 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.017 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-26T22:12:43.019 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.020 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-26T22:12:43.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:43.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.035 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.038 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.042 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-26T22:12:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:43.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:43.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-26T22:12:43.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.050 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.056 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.059 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.060 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.067 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.072 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.075 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.078 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-26T22:12:43.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.084 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-26T22:12:43.101 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:12:43.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:12:43.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.104 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.111 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.112 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.113 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.115 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.118 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.120 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-26T22:12:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-26T22:12:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.169 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.176 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-04-26T22:12:43.177 INFO:teuthology.orchestra.run.smithi012.stdout: "restful", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_schedule" 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.180 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.186 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.187 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-26T22:12:43.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.195 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.196 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:12:43.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.203 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-26T22:12:43.209 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.214 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-26T22:12:43.221 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-26T22:12:43.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-26T22:12:43.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.227 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.228 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:12:43.242 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.248 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.257 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:12:43.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:12:43.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:12:43.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.281 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:12:43.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:12:43.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:12:43.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:12:43.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:12:43.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.302 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.303 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.304 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.305 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.306 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.308 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.309 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.310 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.311 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.312 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.313 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.325 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.326 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.327 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:12:43.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.329 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.330 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.341 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.343 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:43.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:43.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.349 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-26T22:12:43.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.356 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.357 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.360 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.361 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:12:43.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:12:43.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.369 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.370 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.373 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.385 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.396 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.399 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.400 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.402 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.403 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.413 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-26T22:12:43.415 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:12:43.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.417 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:12:43.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:12:43.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:12:43.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:12:43.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:12:43.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.430 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.433 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.435 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.436 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.437 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.438 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.439 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.442 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.446 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-26T22:12:43.451 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.453 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.461 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.463 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.464 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-26T22:12:43.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:43.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.473 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.474 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.486 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.495 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.498 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.499 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.500 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-26T22:12:43.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.508 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-26T22:12:43.520 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.530 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-26T22:12:43.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.544 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.555 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.557 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.558 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.562 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.568 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:12:43.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-26T22:12:43.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.587 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-26T22:12:43.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.593 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.597 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.602 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.605 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.615 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-26T22:12:43.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-26T22:12:43.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-26T22:12:43.623 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-26T22:12:43.624 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-26T22:12:43.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-26T22:12:43.626 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-26T22:12:43.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 95, 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3048838733 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 947904081 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.629 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3456001155 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3090635711 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 836402773 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T22:12:43.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 170039090 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T22:12:43.632 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:12:43.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:42 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/794698310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:12:43.633 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:42 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2962183590' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:12:43.634 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:42 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/794698310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:12:43.634 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:42 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2962183590' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:12:43.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.281+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 msgr2=0x7fb6940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 0x7fb6940a61e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009bd0 tx=0x7fb6a0005f90 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 msgr2=0x7fb6b01a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb698032d80 tx=0x7fb698032db0 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 shutdown_connections 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6b01137a0 0x7fb6b019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b0077780 0x7fb6b01a59b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 --2- 172.21.15.12:0/2962183590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6b0076e60 0x7fb6b01a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 >> 172.21.15.12:0/2962183590 conn(0x7fb6b0100630 msgr2=0x7fb6b0113f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 shutdown_connections 2024-04-26T22:12:43.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:12:42.282+0000 7fb6b6a00640 1 -- 172.21.15.12:0/2962183590 wait complete. 2024-04-26T22:12:43.641 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T22:12:43.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T22:12:43.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:42 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/794698310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:12:43.642 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:42 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2962183590' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:12:43.724 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:43 smithi012 ceph-mon[28184]: pgmap v314: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:43.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:43 smithi077 ceph-mon[32280]: pgmap v314: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:43.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:43 smithi154 ceph-mon[32578]: pgmap v314: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:44.118 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:12:45.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:45 smithi077 ceph-mon[32280]: pgmap v315: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:45.735 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:45 smithi012 ceph-mon[28184]: pgmap v315: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:45.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:45 smithi154 ceph-mon[32578]: pgmap v315: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:46.656 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:12:46.656 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:12:46.656 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-04-26T22:12:46.656 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.004785775, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000981464, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:46.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 222, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 212, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1451232, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 130464, 2024-04-26T22:12:46.658 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.098173958, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.033601264, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.077951125, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.009446687, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 211, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.021612763, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000102430 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:46.659 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1451232, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 6, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 164, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3264, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 185968, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.034809624, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.024561115, 2024-04-26T22:12:46.660 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.004603845, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 162, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.029459314, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000181847 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 185968 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.661 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:12:46.662 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:12:46.663 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 149745, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:12:46.664 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-04-26T22:12:46.665 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 224, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1419946, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 224, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:46.666 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 224, 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1419946, 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:46.667 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:12:47.577 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-26T22:12:47.577 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-26T22:12:47.577 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-26T22:12:47.577 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T22:12:47.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T22:12:47.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:47 smithi012 ceph-mon[28184]: pgmap v316: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:47.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:47 smithi154 ceph-mon[32578]: pgmap v316: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:47.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:47 smithi077 ceph-mon[32280]: pgmap v316: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:48.003 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:12:49.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:49 smithi012 ceph-mon[28184]: pgmap v317: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:49.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:49 smithi154 ceph-mon[32578]: pgmap v317: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:49.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:49 smithi077 ceph-mon[32280]: pgmap v317: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:50.335 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:12:50.602 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:12:50.602 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 229, 2024-04-26T22:12:50.603 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 219, 2024-04-26T22:12:50.603 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1453136, 2024-04-26T22:12:50.603 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 134944, 2024-04-26T22:12:50.603 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-26T22:12:50.603 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.111933072, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.033147885, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.095175239, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.009662339, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 218, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.023507582, 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000107832 2024-04-26T22:12:50.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1453136, 2024-04-26T22:12:50.605 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 134944 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 167, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 186400, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-26T22:12:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.038256881, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.026056260, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.006056365, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 164, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.036481992, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000222451 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:50.607 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 186400 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:12:50.608 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 4, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 2944, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 448, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001276794, 2024-04-26T22:12:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000055443, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001164377, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.001847901, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000615967 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:50.610 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-26T22:12:50.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:12:50.612 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-26T22:12:50.613 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:12:50.614 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148714, 2024-04-26T22:12:50.615 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:12:50.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-26T22:12:50.617 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:12:50.618 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 231, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1420856, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 231, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 231, 2024-04-26T22:12:50.619 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1420856, 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:50.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:12:51.228 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T22:12:51.229 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T22:12:51.229 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T22:12:51.229 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T22:12:51.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T22:12:51.698 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:12:51.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:51 smithi012 ceph-mon[28184]: pgmap v318: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:51.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:51 smithi154 ceph-mon[32578]: pgmap v318: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:51.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:51 smithi077 ceph-mon[32280]: pgmap v318: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:53.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:53 smithi012 ceph-mon[28184]: pgmap v319: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:53.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:53 smithi154 ceph-mon[32578]: pgmap v319: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:53.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:53 smithi077 ceph-mon[32280]: pgmap v319: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T22:12:54.228 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:12:54.228 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:12:54.228 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1890, 2024-04-26T22:12:54.228 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 1877, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 3482750, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 657954, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.430664656, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.148305323, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.171047036, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.102039211, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1877, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.188505010, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000100428 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.229 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1478656, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 136112 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 6, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 169, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1976, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 451578, 2024-04-26T22:12:54.230 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 7, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 5, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.036698421, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.024758338, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.006049174, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000152783, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 166, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.033598228, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000202398 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.231 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 450240 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 767, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 768, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 188976, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 243595, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.166506365, 2024-04-26T22:12:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.056016811, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.041545864, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.052614309, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 766, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.080904242, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000105619 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-04-26T22:12:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4014, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.213701823, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000053239 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-04-26T22:12:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 29, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 13, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 16, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 12, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-04-26T22:12:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 2, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 0, 2024-04-26T22:12:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 4, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 234, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 6, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:12:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 0, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 0 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-04-26T22:12:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:12:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-04-26T22:12:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 334, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-04-26T22:12:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 2, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 20, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 5662663, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 5713248, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 5713112, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 334 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 20, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 23, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 3, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 13, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 13, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 0, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 29, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 29, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 0, 2024-04-26T22:12:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 50284, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 215416 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 0, 2024-04-26T22:12:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 0, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-04-26T22:12:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-04-26T22:12:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 2, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 6, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 4, 2024-04-26T22:12:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 4, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 5326, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:12:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:12:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:12:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 191562, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 63, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-26T22:12:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 9744, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 204, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 103574, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 708, 2024-04-26T22:12:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-04-26T22:12:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 2411, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 26, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 4, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 2411, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2411, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 2.227982205, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000924090 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2411, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 2412 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 2411, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 2411, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 1526, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 882, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 0, 2024-04-26T22:12:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 2, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:12:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-04-26T22:12:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-04-26T22:12:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 103, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 13, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 5, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 5, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 2, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 0 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:12:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osd-slow-ops": { 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 0, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 2659, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 3418891, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 2659, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2659, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 3418891, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2411, 2024-04-26T22:12:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 1518954, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2411, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1518954, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2411, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 2411, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2411, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2411, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x5620dcdee780": { 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:54.283 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:12:55.226 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-26T22:12:55.227 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-26T22:12:55.227 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T22:12:55.227 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T22:12:55.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T22:12:55.709 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:12:55.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:55 smithi012 ceph-mon[28184]: pgmap v320: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:55.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:55 smithi154 ceph-mon[32578]: pgmap v320: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:55.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:55 smithi077 ceph-mon[32280]: pgmap v320: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:57.607 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:57 smithi012 ceph-mon[28184]: pgmap v321: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:57.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:57 smithi154 ceph-mon[32578]: pgmap v321: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-04-26T22:12:57.944 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.004065574, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000926376, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:12:57.945 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 170, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 186928, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.038753387, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.024917506, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.007620971, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:57.946 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 167, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.033354944, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000199730 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 186928 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:12:57.947 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 236, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 224, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1456896, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 137312, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.110600340, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.034179232, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.098768997, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.009621702, 2024-04-26T22:12:57.948 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 224, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.030659449, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000136872 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1456896, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 137312 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-26T22:12:57.949 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:12:57.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:12:57.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-26T22:12:57.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 150776, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 32, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:12:57.953 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-26T22:12:57.954 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 234, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1421246, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 234, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-26T22:12:57.955 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 234, 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1421246, 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T22:12:57.956 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T22:12:57.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:57 smithi077 ceph-mon[32280]: pgmap v321: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:12:58.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T22:12:58.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T22:12:58.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T22:12:58.943 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T22:12:58.943 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T22:12:59.191 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:12:59.637 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:12:59 smithi077 ceph-mon[32280]: pgmap v322: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:59.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:12:59 smithi012 ceph-mon[28184]: pgmap v322: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:59.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:12:59 smithi154 ceph-mon[32578]: pgmap v322: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:01.116 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 3436, 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2381, 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1162129, 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 2526912, 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 16, 2024-04-26T22:13:01.117 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.332422017, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.084318413, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.115947411, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.121439159, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2380, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.303928731, 2024-04-26T22:13:01.118 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000127701 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1890, 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000352255, 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000186 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 682144, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 225360 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1881, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1833, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 497959, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1211375, 2024-04-26T22:13:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 44, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.136458303, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.032608979, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.046194352, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.046607135, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1830, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.225729767, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000123349 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 140, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000024377, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000174 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 43296, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 149808 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2689, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2589, 2024-04-26T22:13:01.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2160158, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1330754, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 11, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.223072572, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.066236211, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.085273525, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.071328897, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2588, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.869879709, 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000336120 2024-04-26T22:13:01.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 386, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000069351, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000179 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1614800, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 491056 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.124 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 915, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.165702532, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000181095 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.125 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4504, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.131012251, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000029087 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 194, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 194, 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-04-26T22:13:01.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 194, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.157583262, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000812284 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 115, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-04-26T22:13:01.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 33, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 9, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 24, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 19, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 3, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 4, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 246, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 238, 2024-04-26T22:13:01.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-04-26T22:13:01.129 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 4, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 234, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:13:01.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 13, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 109, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 155, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 139, 2024-04-26T22:13:01.131 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 6, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 26 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 2, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:13:01.132 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 5, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 3, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 2, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 2, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:13:01.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.134 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 341, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 288, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 288, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 53, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 5, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 20, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 19, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 19, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 1, 2024-04-26T22:13:01.135 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 1, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 1, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 5662663, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 5716446, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 301, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.680167839, 2024-04-26T22:13:01.136 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002259693 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 27, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 30, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 3, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 20, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 20, 2024-04-26T22:13:01.137 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 33, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 39, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 6, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 4, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 25, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 21, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 55516, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 231800 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.138 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 222, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 194, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 37, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7, 2024-04-26T22:13:01.139 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.007810413, 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001115773 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 126, 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.027138402, 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000215384 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-04-26T22:13:01.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.141 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4, 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000863146, 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000215786 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.142 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000316352, 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000158176 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.143 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 11, 2024-04-26T22:13:01.145 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002036032, 2024-04-26T22:13:01.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000185093 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.011609759, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001451219 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 11, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.047118380, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004283489 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000252653, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000252653 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.148 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000765690, 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000382845 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-26T22:13:01.149 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002224897, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001112448 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.150 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.005862944, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002931472 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.020247090, 2024-04-26T22:13:01.151 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004049418 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.001464499, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000732249 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.152 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.153 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023157773, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002894721 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.154 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.006715232, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002238410 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.155 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 6, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 4, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 27, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 13, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 610, 2024-04-26T22:13:01.156 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:01.157 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:01.158 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:01.159 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 343278, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 135, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 616, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 7, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-26T22:13:01.160 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9744, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 204, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 131829, 2024-04-26T22:13:01.161 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 1020, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 5734, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 1766159, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 31, 2024-04-26T22:13:01.162 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 5734, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5734, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 78.993464125, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.013776327 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5734, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 6045 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 5734, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 5, 2024-04-26T22:13:01.163 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 5729, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 97, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 40, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 3, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 304, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 83, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 5125, 2024-04-26T22:13:01.164 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 73, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-04-26T22:13:01.165 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-04-26T22:13:01.166 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 103, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 18, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-04-26T22:13:01.167 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 300, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 20 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-04-26T22:13:01.168 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 3585, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 1, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 4613, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 3, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:01.169 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 8002, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 2995318, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 8002, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:01.170 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 8002, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2995318, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-26T22:13:01.171 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5734, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1766502, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5734, 2024-04-26T22:13:01.172 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1766502, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:01.173 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5734, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 5734, 2024-04-26T22:13:01.174 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5734, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5734, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-26T22:13:01.175 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 3, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 343, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 3, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:01.176 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 3, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 343, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55fb69ba2500": { 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 84, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 341, 2024-04-26T22:13:01.177 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1522142, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 341, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 301, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1522058, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.178 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:01.179 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T22:13:01.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:01 smithi077 ceph-mon[32280]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:13:01.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:01 smithi012 ceph-mon[28184]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:13:01.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:01 smithi154 ceph-mon[32578]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T22:13:01.971 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-26T22:13:01.971 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-26T22:13:01.971 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-26T22:13:01.971 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T22:13:01.971 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T22:13:02.226 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:13:02.636 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:02 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:02.636 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:02 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.636 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:02 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.636 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:02 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.637 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:02 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:02 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:02 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:02 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:02 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:02 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:02 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:02 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:02 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:02 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:02.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:02 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.715 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:03 smithi077 ceph-mon[32280]: pgmap v324: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:13:03.716 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:03 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.716 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:03 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:03 smithi154 ceph-mon[32578]: pgmap v324: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:13:03.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:03 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.892 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:03 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:03 smithi012 ceph-mon[28184]: pgmap v324: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T22:13:03.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:03 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:03 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:03.996 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T22:13:03.996 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:03.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 232, 2024-04-26T22:13:03.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 222, 2024-04-26T22:13:03.996 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1453952, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 136864, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.048069170, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.014262791, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.035307747, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.005820739, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 221, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.012590601, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000056971 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:03.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1453952, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 136864 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 7, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 174, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3424, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 266496, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.012503307, 2024-04-26T22:13:03.998 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.008229934, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.002126010, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 172, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.030421162, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000176867 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-04-26T22:13:03.999 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 266496 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.000249075, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000169052, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:04.000 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.001 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-26T22:13:04.002 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:04.002 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:04.002 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:04.002 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:04.003 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:04.005 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148709, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 31, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:04.006 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:04.007 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 235, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1421273, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 235, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 235, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1421273, 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:04.008 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:04.009 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:04.009 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:04.009 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:04.009 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:04.009 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T22:13:04.746 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-26T22:13:04.746 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-26T22:13:04.746 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-26T22:13:04.746 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T22:13:04.746 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:13:05.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:04 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:13:05.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:13:05.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:04 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:13:05.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-26T22:13:05.391 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:04 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi154 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T22:13:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:13:07.033 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:06 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.033 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:06 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.033 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:06 smithi077 ceph-mon[32280]: pgmap v326: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:06 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:06 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.073 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:06 smithi012 ceph-mon[28184]: pgmap v326: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:07.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:06 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:06 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:07.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:06 smithi154 ceph-mon[32578]: pgmap v326: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T22:13:07.446 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T22:13:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.000970378, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000110472, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 169, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 186752, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.011308139, 2024-04-26T22:13:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.007892941, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001234916, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 166, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.026059297, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000156983 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:13:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 186752 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 234, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 222, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1456352, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 136032, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.045620324, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.014145568, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.033839622, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.005653125, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 222, 2024-04-26T22:13:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.013053827, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000058801 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1456352, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 136032 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-26T22:13:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148692, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 30, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 232, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1420986, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 232, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 232, 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1420986, 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:07.457 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T22:13:08.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:13:08.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:13:08.076 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:07 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:08.116 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:13:08.117 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:07 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:08.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi012 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T22:13:08.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:07 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:08.144 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-26T22:13:08.144 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-26T22:13:08.144 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-26T22:13:08.144 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T22:13:08.144 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T22:13:08.396 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: pgmap v328: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:09.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:08 smithi012 ceph-mon[28184]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:09.140 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: pgmap v328: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:13:09.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:13:09.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:09.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:09.142 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:08 smithi154 ceph-mon[32578]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:09.232 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: pgmap v328: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:13:09.233 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-04-26T22:13:09.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-26T22:13:09.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:09.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:09.234 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:08 smithi077 ceph-mon[32280]: from='mgr.25318 ' entity='mgr.x' 2024-04-26T22:13:10.111 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T22:13:10.111 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:10.111 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:10.111 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:10.111 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.000855711, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000097558, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:10.112 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 6, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 175, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3264, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 187904, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:10.113 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.012223443, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.008208283, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001622263, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 173, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.018306295, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000105816 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.114 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 187904 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 236, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 226, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1455040, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 139424, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.046687663, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.014826521, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.033283842, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.005595212, 2024-04-26T22:13:10.115 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 225, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.013116793, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000058296 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1455040, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 139424 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-04-26T22:13:10.116 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148732, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 31, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:10.119 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:10.120 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 238, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1421766, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 238, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 238, 2024-04-26T22:13:10.121 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1421766, 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T22:13:10.122 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T22:13:10.817 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T22:13:10.818 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T22:13:10.818 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T22:13:10.818 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T22:13:10.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T22:13:11.071 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:13:11.462 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:11 smithi154 ceph-mon[32578]: pgmap v329: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:11.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:11 smithi012 ceph-mon[28184]: pgmap v329: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:11.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:11 smithi077 ceph-mon[32280]: pgmap v329: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-26T22:13:12.848 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-26T22:13:12.848 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:12.848 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.001053602, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000129742, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:12.849 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 2, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 171, 2024-04-26T22:13:12.850 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 187104, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.011350349, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.007835265, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001227120, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 168, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.021900604, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000130360 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:12.851 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 187104 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 237, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 225, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1457168, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 137952, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.045044347, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.014053499, 2024-04-26T22:13:12.852 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.033350958, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.005691348, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 225, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012191042, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000054182 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1457168, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 137952 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.853 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:12.854 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:12.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:12.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148692, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 30, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:12.857 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-26T22:13:12.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 235, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1421376, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 235, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 235, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1421376, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:12.859 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:12.860 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:12.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:12.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:12.860 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-26T22:13:13.527 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:13 smithi154 ceph-mon[32578]: pgmap v330: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:13.554 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-26T22:13:13.554 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-26T22:13:13.554 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-26T22:13:13.554 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T22:13:13.554 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T22:13:13.659 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:13 smithi077 ceph-mon[32280]: pgmap v330: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:13.798 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:13:13.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:13 smithi012 ceph-mon[28184]: pgmap v330: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:15 smithi154 ceph-mon[32578]: pgmap v331: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:15.566 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.001024581, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000196528, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:15.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 6, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 174, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 3264, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 266496, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.011546976, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.007392397, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001834574, 2024-04-26T22:13:15.568 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 172, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.030815547, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000179160 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 266496 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.569 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 234, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 224, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1454496, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 138144, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.047921632, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.014292355, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.036353977, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.005958267, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 223, 2024-04-26T22:13:15.570 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012671152, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000056821 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1454496, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 138144 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-26T22:13:15.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:15.572 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:15.573 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148714, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 30, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:15.575 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:15.576 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 236, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1421506, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 236, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 236, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1421506, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:15.577 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:15.578 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-26T22:13:15.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:15 smithi077 ceph-mon[32280]: pgmap v331: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:15.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:15 smithi012 ceph-mon[28184]: pgmap v331: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:16.303 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-26T22:13:16.304 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-26T22:13:16.304 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-26T22:13:16.304 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T22:13:16.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T22:13:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:13:17.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:17 smithi154 ceph-mon[32578]: pgmap v332: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:17.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:17 smithi154 ceph-mon[32578]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T22:13:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:17 smithi077 ceph-mon[32280]: pgmap v332: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:17.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:17 smithi077 ceph-mon[32280]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T22:13:17.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:17 smithi012 ceph-mon[28184]: pgmap v332: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:17.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:17 smithi012 ceph-mon[28184]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T22:13:18.328 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.001117083, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000193376, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:18.329 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:18.330 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 6, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 172, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 3264, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 187376, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.011332426, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.007455485, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001605782, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 170, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.026141460, 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000153773 2024-04-26T22:13:18.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 187376 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 232, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 222, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1453952, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 136864, 2024-04-26T22:13:18.332 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.044622737, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.013994965, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.032704235, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.005522525, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 221, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.011593625, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000052459 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:18.333 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1453952, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 136864 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:18.334 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:18.335 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:18.336 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148714, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 30, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:18.337 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 234, 2024-04-26T22:13:18.339 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1421246, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 234, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 234, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1421246, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:18.340 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-26T22:13:19.071 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-04-26T22:13:19.071 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-04-26T22:13:19.071 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-04-26T22:13:19.071 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T22:13:19.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid d6d929b6-0416-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T22:13:19.310 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:13:19.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:19 smithi012 ceph-mon[28184]: pgmap v334: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:19.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:19 smithi154 ceph-mon[32578]: pgmap v334: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:19.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:19 smithi077 ceph-mon[32280]: pgmap v334: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:21.006 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.000934144, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000119591, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:21.007 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:13:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 244, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 232, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1459072, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 142432, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.049210206, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.016023808, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.033900017, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.006014375, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 232, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012167246, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000052445 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.009 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1459072, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 142432 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 2, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 176, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 266752, 2024-04-26T22:13:21.010 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.011968729, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.007905624, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001795307, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 173, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.030094028, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000173953 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:21.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 266752 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:13:21.012 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:13:21.013 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:13:21.014 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148692, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 30, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:13:21.015 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 242, 2024-04-26T22:13:21.016 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1422286, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 242, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 242, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1422286, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-26T22:13:21.017 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-26T22:13:21.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:21 smithi012 ceph-mon[28184]: pgmap v335: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:21.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:21 smithi154 ceph-mon[32578]: pgmap v335: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:21.707 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T22:13:21.707 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T22:13:21.708 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T22:13:21.708 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T22:13:21.718 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T22:13:21.718 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:13:21.719 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:13:21.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:21 smithi077 ceph-mon[32280]: pgmap v335: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:13:21.781 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T22:13:21.781 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:13:21.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:13:22.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 -- 172.21.15.12:0/2772150546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c109bc0 msgr2=0x7f0d1c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 --2- 172.21.15.12:0/2772150546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c109bc0 0x7f0d1c10dc00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009860 tx=0x7f0d0802a340 comp rx=0 tx=0).stop 2024-04-26T22:13:22.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 -- 172.21.15.12:0/2772150546 shutdown_connections 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 --2- 172.21.15.12:0/2772150546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c10e2e0 0x7f0d1c115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 --2- 172.21.15.12:0/2772150546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c109bc0 0x7f0d1c10dc00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 --2- 172.21.15.12:0/2772150546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 -- 172.21.15.12:0/2772150546 >> 172.21.15.12:0/2772150546 conn(0x7f0d1c076f50 msgr2=0x7f0d1c079370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 -- 172.21.15.12:0/2772150546 shutdown_connections 2024-04-26T22:13:22.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.058+0000 7f0d20a00640 1 -- 172.21.15.12:0/2772150546 wait complete. 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.059+0000 7f0d20a00640 1 Processor -- start 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.059+0000 7f0d20a00640 1 -- start start 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c10e2e0 0x7f0d1c1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 0x7f0d1c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c119250 con 0x7f0d1c109210 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c1190d0 con 0x7f0d1c10e2e0 2024-04-26T22:13:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d20a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c1193d0 con 0x7f0d1c19f5f0 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d19e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d19e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45266/0 (socket says 172.21.15.12:45266) 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d1a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 0x7f0d1c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.060+0000 7f0d19e00640 1 -- 172.21.15.12:0/4187239527 learned_addr learned my addr 172.21.15.12:0/4187239527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19400640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c10e2e0 0x7f0d1c1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 -- 172.21.15.12:0/4187239527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 msgr2=0x7f0d1c19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 0x7f0d1c19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 -- 172.21.15.12:0/4187239527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c10e2e0 msgr2=0x7f0d1c1a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c10e2e0 0x7f0d1c1a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 -- 172.21.15.12:0/4187239527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1a0110 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d1a800640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 0x7f0d1c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d19e00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00ea60 tx=0x7f0d0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00cde0 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.061+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c0035f0 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.062+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1a0400 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.062+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c010690 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.062+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1ac650 con 0x7f0d1c109210 2024-04-26T22:13:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.063+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c069c10 con 0x7f0d1c109210 2024-04-26T22:13:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.065+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0d0c0037a0 con 0x7f0d1c109210 2024-04-26T22:13:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.066+0000 7f0d12a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 0x7f0d000a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.066+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f0d0c0d0f00 con 0x7f0d1c109210 2024-04-26T22:13:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.067+0000 7f0d19400640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 0x7f0d000a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.067+0000 7f0d19400640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 0x7f0d000a6220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d080303f0 tx=0x7f0d0802a840 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.071+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0d0c09d140 con 0x7f0d1c109210 2024-04-26T22:13:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.270+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d1c1a0b20 con 0x7f0d1c109210 2024-04-26T22:13:22.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.272+0000 7f0d12a00640 1 -- 172.21.15.12:0/4187239527 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25344 (secure 0 0 0) 0x7f0d0c09ca40 con 0x7f0d1c109210 2024-04-26T22:13:22.272 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:13:22.273 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":105,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:13:16.884039+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1038496765":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/3660687894":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/36130226":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/53237046":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/505816482":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2706265351":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/3660687894":"2024-04-27T22:02:35.907627+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 msgr2=0x7f0d000a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 0x7f0d000a6220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d080303f0 tx=0x7f0d0802a840 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 msgr2=0x7f0d1c1a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00ea60 tx=0x7f0d0c00ef20 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 shutdown_connections 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f0d000a3d60 0x7f0d000a6220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d1c19f5f0 0x7f0d1c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d1c10e2e0 0x7f0d1c1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 --2- 172.21.15.12:0/4187239527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d1c109210 0x7f0d1c1a5440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 >> 172.21.15.12:0/4187239527 conn(0x7f0d1c076f50 msgr2=0x7f0d1c0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 shutdown_connections 2024-04-26T22:13:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.275+0000 7f0d20a00640 1 -- 172.21.15.12:0/4187239527 wait complete. 2024-04-26T22:13:22.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:54:29.133630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:59:41.200542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:59:42.493543+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T21:59:46.121636+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T22:13:22.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:13:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 -- 172.21.15.12:0/2559298680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 msgr2=0x7faec010f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 --2- 172.21.15.12:0/2559298680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec010f2d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009860 tx=0x7faeb002a3d0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 -- 172.21.15.12:0/2559298680 shutdown_connections 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 --2- 172.21.15.12:0/2559298680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0109870 0x7faec0115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 --2- 172.21.15.12:0/2559298680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0108ed0 0x7faec0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 --2- 172.21.15.12:0/2559298680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec010f2d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 -- 172.21.15.12:0/2559298680 >> 172.21.15.12:0/2559298680 conn(0x7faec0078750 msgr2=0x7faec0107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 -- 172.21.15.12:0/2559298680 shutdown_connections 2024-04-26T22:13:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.510+0000 7faebf200640 1 -- 172.21.15.12:0/2559298680 wait complete. 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.511+0000 7faebf200640 1 Processor -- start 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 -- start start 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0108ed0 0x7faec019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec01a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faec01190d0 con 0x7faec0109870 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faec0118f50 con 0x7faec0108ed0 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faec0119250 con 0x7faec010eef0 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faeb7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faeb7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45276/0 (socket says 172.21.15.12:45276) 2024-04-26T22:13:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faeb7e00640 1 -- 172.21.15.12:0/1347458752 learned_addr learned my addr 172.21.15.12:0/1347458752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebde00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0108ed0 0x7faec019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.512+0000 7faebe800640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec01a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 -- 172.21.15.12:0/1347458752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 msgr2=0x7faec01a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec01a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 -- 172.21.15.12:0/1347458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0108ed0 msgr2=0x7faec019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0108ed0 0x7faec019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 -- 172.21.15.12:0/1347458752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec01a4580 con 0x7faec0109870 2024-04-26T22:13:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb7e00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faea800e9c0 tx=0x7faea800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea800ccc0 con 0x7faec0109870 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea80054d0 con 0x7faec0109870 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8010430 con 0x7faec0109870 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.513+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec01a4870 con 0x7faec0109870 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.514+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec01ac100 con 0x7faec0109870 2024-04-26T22:13:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.515+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec0069ac0 con 0x7faec0109870 2024-04-26T22:13:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.517+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7faea8004f80 con 0x7faec0109870 2024-04-26T22:13:22.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.518+0000 7faeb6a00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 0x7faea40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.519+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7faea80d1040 con 0x7faec0109870 2024-04-26T22:13:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.519+0000 7faebde00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 0x7faea40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.519+0000 7faebde00640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 0x7faea40a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faeb0032ff0 tx=0x7faeb0005c90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.522+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faea809d280 con 0x7faec0109870 2024-04-26T22:13:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:22 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/4187239527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:22.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:22 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/4187239527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.730+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faec0110d50 con 0x7faec0109870 2024-04-26T22:13:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.731+0000 7faeb6a00640 1 -- 172.21.15.12:0/1347458752 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7faea809cb80 con 0x7faec0109870 2024-04-26T22:13:22.731 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-26T22:13:22.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:22 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/4187239527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.733+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 msgr2=0x7faea40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.733+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 0x7faea40a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faeb0032ff0 tx=0x7faeb0005c90 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.733+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 msgr2=0x7faec019fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.733+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faea800e9c0 tx=0x7faea800ee80 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 shutdown_connections 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7faea40a3d20 0x7faea40a61e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faec010eef0 0x7faec01a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec0109870 0x7faec019fad0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 --2- 172.21.15.12:0/1347458752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faec0108ed0 0x7faec019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 >> 172.21.15.12:0/1347458752 conn(0x7faec0078750 msgr2=0x7faec0079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 shutdown_connections 2024-04-26T22:13:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.734+0000 7faebf200640 1 -- 172.21.15.12:0/1347458752 wait complete. 2024-04-26T22:13:22.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:13:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 -- 172.21.15.12:0/83823248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 msgr2=0x7f5a58077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 --2- 172.21.15.12:0/83823248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a58077240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009900 tx=0x7f5a4802a400 comp rx=0 tx=0).stop 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 -- 172.21.15.12:0/83823248 shutdown_connections 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 --2- 172.21.15.12:0/83823248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58113680 0x7f5a58115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 --2- 172.21.15.12:0/83823248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58077780 0x7f5a58113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 --2- 172.21.15.12:0/83823248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a58077240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 -- 172.21.15.12:0/83823248 >> 172.21.15.12:0/83823248 conn(0x7f5a58100470 msgr2=0x7f5a58102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 -- 172.21.15.12:0/83823248 shutdown_connections 2024-04-26T22:13:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.977+0000 7f5a5da00640 1 -- 172.21.15.12:0/83823248 wait complete. 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.978+0000 7f5a5da00640 1 Processor -- start 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 -- start start 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58077780 0x7f5a5819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 0x7f5a581a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a581190a0 con 0x7f5a58076e60 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58118f20 con 0x7f5a58113680 2024-04-26T22:13:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a5da00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58119220 con 0x7f5a58077780 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.979+0000 7f5a57e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45286/0 (socket says 172.21.15.12:45286) 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 -- 172.21.15.12:0/3723735929 learned_addr learned my addr 172.21.15.12:0/3723735929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a4fe00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58077780 0x7f5a5819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a5d000640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 0x7f5a581a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 -- 172.21.15.12:0/3723735929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58077780 msgr2=0x7f5a5819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58077780 0x7f5a5819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 -- 172.21.15.12:0/3723735929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 msgr2=0x7f5a581a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 0x7f5a581a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 -- 172.21.15.12:0/3723735929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a581a4690 con 0x7f5a58076e60 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a5d000640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 0x7f5a581a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.980+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006010 tx=0x7f5a480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.981+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4803f030 con 0x7f5a58076e60 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.981+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a581a4920 con 0x7f5a58076e60 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.981+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a480044c0 con 0x7f5a58076e60 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.981+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48035070 con 0x7f5a58076e60 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.981+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a581ac210 con 0x7f5a58076e60 2024-04-26T22:13:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.982+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58069670 con 0x7f5a58076e60 2024-04-26T22:13:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.984+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5a48042030 con 0x7f5a58076e60 2024-04-26T22:13:22.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.986+0000 7f5a55600640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:22.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.986+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f5a480f1090 con 0x7f5a58076e60 2024-04-26T22:13:22.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.986+0000 7f5a4fe00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:22.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.987+0000 7f5a4fe00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a400095f0 tx=0x7f5a40005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:22.990+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5a480bcce0 con 0x7f5a58076e60 2024-04-26T22:13:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.194+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a5810e810 con 0x7f5a58076e60 2024-04-26T22:13:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.195+0000 7f5a55600640 1 -- 172.21.15.12:0/3723735929 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f5a480c0fe0 con 0x7f5a58076e60 2024-04-26T22:13:23.195 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 msgr2=0x7f5a3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a400095f0 tx=0x7f5a40005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 msgr2=0x7f5a5819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006010 tx=0x7f5a480040c0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 shutdown_connections 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a58113680 0x7f5a581a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a58077780 0x7f5a5819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 --2- 172.21.15.12:0/3723735929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a58076e60 0x7f5a5819f6f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 >> 172.21.15.12:0/3723735929 conn(0x7f5a58100470 msgr2=0x7f5a58102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:23.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 shutdown_connections 2024-04-26T22:13:23.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.198+0000 7f5a5da00640 1 -- 172.21.15.12:0/3723735929 wait complete. 2024-04-26T22:13:23.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:13:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.441+0000 7ff4a9800640 1 -- 172.21.15.12:0/266395905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a410ef10 msgr2=0x7ff4a4111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.441+0000 7ff4a9800640 1 --2- 172.21.15.12:0/266395905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a410ef10 0x7ff4a4111360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009ca0 tx=0x7ff49c02a950 comp rx=0 tx=0).stop 2024-04-26T22:13:23.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 -- 172.21.15.12:0/266395905 shutdown_connections 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 --2- 172.21.15.12:0/266395905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a410ef10 0x7ff4a4111360 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 --2- 172.21.15.12:0/266395905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 0x7ff4a410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 --2- 172.21.15.12:0/266395905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a4074d60 0x7ff4a4075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 -- 172.21.15.12:0/266395905 >> 172.21.15.12:0/266395905 conn(0x7ff4a40fe200 msgr2=0x7ff4a4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 -- 172.21.15.12:0/266395905 shutdown_connections 2024-04-26T22:13:23.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.442+0000 7ff4a9800640 1 -- 172.21.15.12:0/266395905 wait complete. 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.443+0000 7ff4a9800640 1 Processor -- start 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 -- start start 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 0x7ff4a419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a410ef10 0x7ff4a419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4114b60 con 0x7ff4a4074d60 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a41149e0 con 0x7ff4a4075680 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a9800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a4114ce0 con 0x7ff4a410ef10 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45302/0 (socket says 172.21.15.12:45302) 2024-04-26T22:13:23.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.444+0000 7ff4a3e00640 1 -- 172.21.15.12:0/1779221487 learned_addr learned my addr 172.21.15.12:0/1779221487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a8e00640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a410ef10 0x7ff4a419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3400640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 0x7ff4a419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 -- 172.21.15.12:0/1779221487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a410ef10 msgr2=0x7ff4a419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a410ef10 0x7ff4a419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 -- 172.21.15.12:0/1779221487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 msgr2=0x7ff4a419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 0x7ff4a419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 -- 172.21.15.12:0/1779221487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a41a0060 con 0x7ff4a4074d60 2024-04-26T22:13:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a3e00640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9f0 tx=0x7ff49400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.445+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49400cd80 con 0x7ff4a4074d60 2024-04-26T22:13:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.446+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a41a0350 con 0x7ff4a4074d60 2024-04-26T22:13:23.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.446+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4940108e0 con 0x7ff4a4074d60 2024-04-26T22:13:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.446+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494018980 con 0x7ff4a4074d60 2024-04-26T22:13:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.446+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a41a7be0 con 0x7ff4a4074d60 2024-04-26T22:13:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.447+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a4069ac0 con 0x7ff4a4074d60 2024-04-26T22:13:23.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.449+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff494010430 con 0x7ff4a4074d60 2024-04-26T22:13:23.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.451+0000 7ff4a0c00640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 0x7ff4880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.451+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7ff4940d0ed0 con 0x7ff4a4074d60 2024-04-26T22:13:23.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.451+0000 7ff4a3400640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 0x7ff4880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.452+0000 7ff4a3400640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 0x7ff4880a6260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4900095f0 tx=0x7ff490005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:23.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.456+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff49409cac0 con 0x7ff4a4074d60 2024-04-26T22:13:23.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:23 smithi077 ceph-mon[32280]: pgmap v336: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:23.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1347458752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:23 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3723735929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:23 smithi012 ceph-mon[28184]: pgmap v336: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:23.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1347458752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:23 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3723735929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:23 smithi154 ceph-mon[32578]: pgmap v336: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:23.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1347458752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:23 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3723735929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:13:23.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.661+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4a402dad0 con 0x7ff4a4074d60 2024-04-26T22:13:23.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.662+0000 7ff4a0c00640 1 -- 172.21.15.12:0/1779221487 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7ff4940a0dc0 con 0x7ff4a4074d60 2024-04-26T22:13:23.662 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 msgr2=0x7ff4880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 0x7ff4880a6260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4900095f0 tx=0x7ff490005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 msgr2=0x7ff4a419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff49400e9f0 tx=0x7ff49400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 shutdown_connections 2024-04-26T22:13:23.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7ff4880a3da0 0x7ff4880a6260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a410ef10 0x7ff4a419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a4075680 0x7ff4a419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 --2- 172.21.15.12:0/1779221487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4a4074d60 0x7ff4a419b0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 >> 172.21.15.12:0/1779221487 conn(0x7ff4a40fe200 msgr2=0x7ff4a40ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 shutdown_connections 2024-04-26T22:13:23.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.665+0000 7ff4a9800640 1 -- 172.21.15.12:0/1779221487 wait complete. 2024-04-26T22:13:23.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T22:13:23.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 -- 172.21.15.12:0/661986101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb8104de0 msgr2=0x7f8bb81051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 --2- 172.21.15.12:0/661986101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb8104de0 0x7f8bb81051c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009860 tx=0x7f8ba802a340 comp rx=0 tx=0).stop 2024-04-26T22:13:23.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 -- 172.21.15.12:0/661986101 shutdown_connections 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 --2- 172.21.15.12:0/661986101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8109eb0 0x7f8bb8111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 --2- 172.21.15.12:0/661986101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8105790 0x7f8bb81097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 --2- 172.21.15.12:0/661986101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb8104de0 0x7f8bb81051c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 -- 172.21.15.12:0/661986101 >> 172.21.15.12:0/661986101 conn(0x7f8bb81006c0 msgr2=0x7f8bb8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 -- 172.21.15.12:0/661986101 shutdown_connections 2024-04-26T22:13:23.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.905+0000 7f8bbce00640 1 -- 172.21.15.12:0/661986101 wait complete. 2024-04-26T22:13:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.906+0000 7f8bbce00640 1 Processor -- start 2024-04-26T22:13:23.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.906+0000 7f8bbce00640 1 -- start start 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 0x7f8bb819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb819fce0 0x7f8bb81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8114e10 con 0x7f8bb8104de0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8114c90 con 0x7f8bb8109eb0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bbce00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8114f90 con 0x7f8bb819fce0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bb5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bb5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45316/0 (socket says 172.21.15.12:45316) 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bb5e00640 1 -- 172.21.15.12:0/2811161645 learned_addr learned my addr 172.21.15.12:0/2811161645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bb5400640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 0x7f8bb819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.907+0000 7f8bb6800640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb819fce0 0x7f8bb81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 -- 172.21.15.12:0/2811161645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb819fce0 msgr2=0x7f8bb81a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb819fce0 0x7f8bb81a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 -- 172.21.15.12:0/2811161645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 msgr2=0x7f8bb819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 0x7f8bb819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 -- 172.21.15.12:0/2811161645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a07d0 con 0x7f8bb8104de0 2024-04-26T22:13:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5400640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 0x7f8bb819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bb5e00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8035040 tx=0x7f8ba8032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8037040 con 0x7f8bb8104de0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba8041070 con 0x7f8bb8104de0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a0a60 con 0x7f8bb8104de0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.908+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003850 con 0x7f8bb8104de0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.909+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81a81c0 con 0x7f8bb8104de0 2024-04-26T22:13:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.910+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb8069ba0 con 0x7f8bb8104de0 2024-04-26T22:13:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.912+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8ba80039f0 con 0x7f8bb8104de0 2024-04-26T22:13:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.913+0000 7f8baea00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 0x7f8b9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.913+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f8ba80f0600 con 0x7f8bb8104de0 2024-04-26T22:13:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.914+0000 7f8bb5400640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 0x7f8b9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:23.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.914+0000 7f8bb5400640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 0x7f8b9c0a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00095f0 tx=0x7f8ba0005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:23.918+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8ba80bc950 con 0x7f8bb8104de0 2024-04-26T22:13:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.127+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8bb810d0e0 con 0x7f8bb8104de0 2024-04-26T22:13:24.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.128+0000 7f8baea00640 1 -- 172.21.15.12:0/2811161645 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f8ba80bc250 con 0x7f8bb8104de0 2024-04-26T22:13:24.128 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 msgr2=0x7f8b9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 0x7f8b9c0a61e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00095f0 tx=0x7f8ba0005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 msgr2=0x7f8bb819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8035040 tx=0x7f8ba8032d50 comp rx=0 tx=0).stop 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 shutdown_connections 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8b9c0a3d20 0x7f8b9c0a61e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb819fce0 0x7f8bb81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb8109eb0 0x7f8bb819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 --2- 172.21.15.12:0/2811161645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bb8104de0 0x7f8bb819b470 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 >> 172.21.15.12:0/2811161645 conn(0x7f8bb81006c0 msgr2=0x7f8bb8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 shutdown_connections 2024-04-26T22:13:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.131+0000 7f8bbce00640 1 -- 172.21.15.12:0/2811161645 wait complete. 2024-04-26T22:13:24.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:13:24.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1378959348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 msgr2=0x7f8e9c077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1378959348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c077240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009930 tx=0x7f8e9002a440 comp rx=0 tx=0).stop 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1378959348 shutdown_connections 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1378959348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1378959348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c1130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1378959348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c077240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1378959348 >> 172.21.15.12:0/1378959348 conn(0x7f8e9c1003d0 msgr2=0x7f8e9c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1378959348 shutdown_connections 2024-04-26T22:13:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.371+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1378959348 wait complete. 2024-04-26T22:13:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 Processor -- start 2024-04-26T22:13:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 -- start start 2024-04-26T22:13:24.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c119030 con 0x7f8e9c1135e0 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c118eb0 con 0x7f8e9c077780 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.373+0000 7f8ea2e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1191b0 con 0x7f8e9c076e60 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea1000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45330/0 (socket says 172.21.15.12:45330) 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 -- 172.21.15.12:0/1896957623 learned_addr learned my addr 172.21.15.12:0/1896957623 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:24.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea1a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 -- 172.21.15.12:0/1896957623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 msgr2=0x7f8e9c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 -- 172.21.15.12:0/1896957623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 msgr2=0x7f8e9c19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 -- 172.21.15.12:0/1896957623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1a4550 con 0x7f8e9c1135e0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea1000640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.374+0000 7f8ea2400640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400eb40 tx=0x7f8e9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.375+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94004b20 con 0x7f8e9c1135e0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.375+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e94004cc0 con 0x7f8e9c1135e0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.375+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c1a4840 con 0x7f8e9c1135e0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.375+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94010720 con 0x7f8e9c1135e0 2024-04-26T22:13:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.375+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1ac0d0 con 0x7f8e9c1135e0 2024-04-26T22:13:24.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.376+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c069670 con 0x7f8e9c1135e0 2024-04-26T22:13:24.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.378+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8e94005560 con 0x7f8e9c1135e0 2024-04-26T22:13:24.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.380+0000 7f8e9a000640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 0x7f8e840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.381+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f8e940d12e0 con 0x7f8e9c1135e0 2024-04-26T22:13:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.381+0000 7f8ea1a00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 0x7f8e840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.382+0000 7f8ea1a00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 0x7f8e840a61e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e90031440 tx=0x7f8e90005a20 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:24.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.385+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e9409d580 con 0x7f8e9c1135e0 2024-04-26T22:13:24.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1779221487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:24 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2811161645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.608+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e9c02dad0 con 0x7f8e9c1135e0 2024-04-26T22:13:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.609+0000 7f8e9a000640 1 -- 172.21.15.12:0/1896957623 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v105) ==== 81+0+998 (secure 0 0 0) 0x7f8e9409ce80 con 0x7f8e9c1135e0 2024-04-26T22:13:24.609 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1038496765 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.609 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/600788188 2024-04-27T21:51:07.889475+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1377681366 2024-04-27T21:51:07.889475+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1310858189 2024-04-27T21:51:07.889475+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/3660687894 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/36130226 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3243088974 2024-04-27T21:51:07.889475+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3387900079 2024-04-27T21:51:42.978579+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/53237046 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/505816482 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2638555806 2024-04-27T21:51:42.978579+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2513368485 2024-04-27T21:51:42.978579+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4197187136 2024-04-27T21:51:42.978579+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2706265351 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.610 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/3660687894 2024-04-27T22:02:35.907627+0000 2024-04-26T22:13:24.611 INFO:teuthology.orchestra.run.smithi012.stderr:listed 17 entries 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 msgr2=0x7f8e840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 0x7f8e840a61e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e90031440 tx=0x7f8e90005a20 comp rx=0 tx=0).stop 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 msgr2=0x7f8e9c1a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400eb40 tx=0x7f8e9400c2d0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 shutdown_connections 2024-04-26T22:13:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f8e840a3d20 0x7f8e840a61e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e9c1135e0 0x7f8e9c1a3e20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c077780 0x7f8e9c19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 --2- 172.21.15.12:0/1896957623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e9c076e60 0x7f8e9c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 >> 172.21.15.12:0/1896957623 conn(0x7f8e9c1003d0 msgr2=0x7f8e9c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 shutdown_connections 2024-04-26T22:13:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:24.612+0000 7f8ea2e00640 1 -- 172.21.15.12:0/1896957623 wait complete. 2024-04-26T22:13:24.630 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T22:13:24.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:13:24.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T22:13:24.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1779221487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:24 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2811161645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1779221487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:24 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2811161645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T22:13:24.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:13:24.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:13:24.748 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:13:24.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:13:24.802 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-26T22:13:24.843 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:13:24.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:13:24.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:13:24.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:13:24.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-26T22:13:25.050 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T22:13:25.061 INFO:tasks.cephadm:Teardown begin 2024-04-26T22:13:25.061 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T22:13:25.062 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:13:25.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 -- 172.21.15.12:0/17065310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc200109950 msgr2=0x7fc20010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/17065310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc200109950 0x7fc20010d980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009900 tx=0x7fc1ec02a400 comp rx=0 tx=0).stop 2024-04-26T22:13:25.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 -- 172.21.15.12:0/17065310 shutdown_connections 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/17065310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc20010e060 0x7fc200115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/17065310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc200109950 0x7fc20010d980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/17065310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 0x7fc200109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 -- 172.21.15.12:0/17065310 >> 172.21.15.12:0/17065310 conn(0x7fc2000784b0 msgr2=0x7fc20007a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 -- 172.21.15.12:0/17065310 shutdown_connections 2024-04-26T22:13:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.248+0000 7fc1ffe00640 1 -- 172.21.15.12:0/17065310 wait complete. 2024-04-26T22:13:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.249+0000 7fc1ffe00640 1 Processor -- start 2024-04-26T22:13:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.249+0000 7fc1ffe00640 1 -- start start 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 0x7fc20019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2001a3d80 0x7fc2001a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc200118fd0 con 0x7fc20010e060 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc200118e50 con 0x7fc200108fa0 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ffe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc200119150 con 0x7fc2001a3d80 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45342/0 (socket says 172.21.15.12:45342) 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 -- 172.21.15.12:0/2658668376 learned_addr learned my addr 172.21.15.12:0/2658668376 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fea00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 0x7fc20019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1ff400640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2001a3d80 0x7fc2001a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 -- 172.21.15.12:0/2658668376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2001a3d80 msgr2=0x7fc2001a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2001a3d80 0x7fc2001a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 -- 172.21.15.12:0/2658668376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 msgr2=0x7fc20019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 0x7fc20019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.250+0000 7fc1fe000640 1 -- 172.21.15.12:0/2658668376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2001a4870 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1fe000640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005ab0 tx=0x7fc1ec003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec03f070 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ec031d60 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec035030 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2001a4b00 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.251+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001ac320 con 0x7fc20010e060 2024-04-26T22:13:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.253+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc200069ac0 con 0x7fc20010e060 2024-04-26T22:13:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.255+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc1ec042070 con 0x7fc20010e060 2024-04-26T22:13:25.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.257+0000 7fc1f7400640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 0x7fc1e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.257+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7fc1ec0f1340 con 0x7fc20010e060 2024-04-26T22:13:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.257+0000 7fc1fea00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 0x7fc1e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.258+0000 7fc1fea00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 0x7fc1e40a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000bbe0 tx=0x7fc1f000b390 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.262+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc1ec0bcf30 con 0x7fc20010e060 2024-04-26T22:13:25.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.513+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc200109950 con 0x7fc20010e060 2024-04-26T22:13:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.515+0000 7fc1f7400640 1 -- 172.21.15.12:0/2658668376 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7fc1ec0c1230 con 0x7fc20010e060 2024-04-26T22:13:25.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:13:25.516 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:13:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.518+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 msgr2=0x7fc1e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.518+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 0x7fc1e40a6260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000bbe0 tx=0x7fc1f000b390 comp rx=0 tx=0).stop 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.518+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 msgr2=0x7fc20019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.518+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec005ab0 tx=0x7fc1ec003cd0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 shutdown_connections 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fc1e40a3da0 0x7fc1e40a6260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2001a3d80 0x7fc2001a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc20010e060 0x7fc20019fa50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 --2- 172.21.15.12:0/2658668376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc200108fa0 0x7fc20019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 >> 172.21.15.12:0/2658668376 conn(0x7fc2000784b0 msgr2=0x7fc200079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 shutdown_connections 2024-04-26T22:13:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.519+0000 7fc1ffe00640 1 -- 172.21.15.12:0/2658668376 wait complete. 2024-04-26T22:13:25.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:13:25.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:25 smithi012 ceph-mon[28184]: pgmap v337: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:25.612 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:25 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1896957623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:13:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:25 smithi154 ceph-mon[32578]: pgmap v337: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:25.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:25 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1896957623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:13:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:25 smithi077 ceph-mon[32280]: pgmap v337: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:25.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:25 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1896957623' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 -- 172.21.15.12:0/165339348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c104920 msgr2=0x7f1f0c104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 --2- 172.21.15.12:0/165339348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c104920 0x7f1f0c104da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009890 tx=0x7f1ef802a360 comp rx=0 tx=0).stop 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 -- 172.21.15.12:0/165339348 shutdown_connections 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 --2- 172.21.15.12:0/165339348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c1117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 --2- 172.21.15.12:0/165339348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c104920 0x7f1f0c104da0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 --2- 172.21.15.12:0/165339348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c10ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 -- 172.21.15.12:0/165339348 >> 172.21.15.12:0/165339348 conn(0x7f1f0c1003d0 msgr2=0x7f1f0c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 -- 172.21.15.12:0/165339348 shutdown_connections 2024-04-26T22:13:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.755+0000 7f1f10e00640 1 -- 172.21.15.12:0/165339348 wait complete. 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 Processor -- start 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 -- start start 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 0x7f1f0c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c114bc0 con 0x7f1f0c1052e0 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c114a40 con 0x7f1f0c10a930 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.756+0000 7f1f10e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c114d40 con 0x7f1f0c104920 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45360/0 (socket says 172.21.15.12:45360) 2024-04-26T22:13:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0be00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 -- 172.21.15.12:0/3305042005 learned_addr learned my addr 172.21.15.12:0/3305042005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 0x7f1f0c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 -- 172.21.15.12:0/3305042005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 msgr2=0x7f1f0c19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 0x7f1f0c19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 -- 172.21.15.12:0/3305042005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 msgr2=0x7f1f0c19fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c19fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0aa00640 1 -- 172.21.15.12:0/3305042005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c1a0150 con 0x7f1f0c1052e0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0be00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.757+0000 7f1f0b400640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 0x7f1f0c19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f0aa00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009860 tx=0x7f1ef8032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8040020 con 0x7f1f0c1052e0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8030d20 con 0x7f1f0c1052e0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c1a03e0 con 0x7f1f0c1052e0 2024-04-26T22:13:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8037040 con 0x7f1f0c1052e0 2024-04-26T22:13:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.758+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c1a7cd0 con 0x7f1f0c1052e0 2024-04-26T22:13:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.760+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c069ac0 con 0x7f1f0c1052e0 2024-04-26T22:13:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.761+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1ef8043020 con 0x7f1f0c1052e0 2024-04-26T22:13:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.763+0000 7f1f03e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 0x7f1ef00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.763+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f1ef80f0780 con 0x7f1f0c1052e0 2024-04-26T22:13:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.763+0000 7f1f0b400640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 0x7f1ef00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.764+0000 7f1f0b400640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 0x7f1ef00a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005fd0 tx=0x7f1efc009260 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:25.767+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ef804a020 con 0x7f1f0c1052e0 2024-04-26T22:13:26.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.016+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f0c19c4b0 con 0x7f1f0c1052e0 2024-04-26T22:13:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.017+0000 7f1f03e00640 1 -- 172.21.15.12:0/3305042005 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11800 (secure 0 0 0) 0x7f1ef80bcad0 con 0x7f1f0c1052e0 2024-04-26T22:13:26.017 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:13:26.017 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/2930574103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2930574103},{"type":"v1","addr":"172.21.15.12:6835","nonce":2930574103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/247823794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":247823794},{"type":"v1","addr":"172.21.15.12:6837","nonce":247823794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/1291467894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":1291467894},{"type":"v1","addr":"172.21.15.12:6841","nonce":1291467894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/432973357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":432973357},{"type":"v1","addr":"172.21.15.77:6833","nonce":432973357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/1839140447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1839140447},{"type":"v1","addr":"172.21.15.154:6833","nonce":1839140447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3868639549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3868639549},{"type":"v1","addr":"172.21.15.77:6835","nonce":3868639549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1762309874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1762309874},{"type":"v1","addr":"172.21.15.154:6835","nonce":1762309874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1703155205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1703155205},{"type":"v1","addr":"172.21.15.77:6837","nonce":1703155205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/175106835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":175106835},{"type":"v1","addr":"172.21.15.154:6837","nonce":175106835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/404103825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":404103825},{"type":"v1","addr":"172.21.15.154:6839","nonce":404103825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:59:43.592535+0000","modified":"2024-04-26T22:00:09.736381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/3734088968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3734088968},{"type":"v1","addr":"172.21.15.12:6839","nonce":3734088968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6839/4217479283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":4217479283},{"type":"v1","addr":"172.21.15.77:6839","nonce":4217479283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-26T22:13:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 msgr2=0x7f1ef00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 0x7f1ef00a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005fd0 tx=0x7f1efc009260 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 msgr2=0x7f1f0c19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009860 tx=0x7f1ef8032d20 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 shutdown_connections 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f1ef00a3d20 0x7f1ef00a61e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0c10a930 0x7f1f0c19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1052e0 0x7f1f0c19b6a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 --2- 172.21.15.12:0/3305042005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0c104920 0x7f1f0c19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 >> 172.21.15.12:0/3305042005 conn(0x7f1f0c1003d0 msgr2=0x7f1f0c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 shutdown_connections 2024-04-26T22:13:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.020+0000 7f1f10e00640 1 -- 172.21.15.12:0/3305042005 wait complete. 2024-04-26T22:13:26.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 -- 172.21.15.12:0/1650798771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4104510 msgr2=0x7f6cb41048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/1650798771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4104510 0x7f6cb41048f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009860 tx=0x7f6ca402a340 comp rx=0 tx=0).stop 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 -- 172.21.15.12:0/1650798771 shutdown_connections 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/1650798771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4101630 0x7f6cb410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/1650798771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4100c90 0x7f6cb41010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/1650798771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4104510 0x7f6cb41048f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 -- 172.21.15.12:0/1650798771 >> 172.21.15.12:0/1650798771 conn(0x7f6cb4077c20 msgr2=0x7f6cb40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 -- 172.21.15.12:0/1650798771 shutdown_connections 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.253+0000 7f6cb3e00640 1 -- 172.21.15.12:0/1650798771 wait complete. 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.255+0000 7f6cb3e00640 1 Processor -- start 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.255+0000 7f6cb3e00640 1 -- start start 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.255+0000 7f6cb3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.255+0000 7f6cb3e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 0x7f6cb419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4104510 0x7f6cb419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb3e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4112a20 con 0x7f6cb4100c90 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb3e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41128a0 con 0x7f6cb4101630 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb3e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4112ba0 con 0x7f6cb4104510 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45370/0 (socket says 172.21.15.12:45370) 2024-04-26T22:13:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 -- 172.21.15.12:0/3617579341 learned_addr learned my addr 172.21.15.12:0/3617579341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 0x7f6cb419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb3400640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4104510 0x7f6cb419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 -- 172.21.15.12:0/3617579341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4104510 msgr2=0x7f6cb419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4104510 0x7f6cb419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 -- 172.21.15.12:0/3617579341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 msgr2=0x7f6cb419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 0x7f6cb419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2a00640 1 -- 172.21.15.12:0/3617579341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41a00b0 con 0x7f6cb4100c90 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.256+0000 7f6cb2000640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 0x7f6cb419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cb2a00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009980 tx=0x7f6ca4032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4040020 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a03a0 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4030dc0 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4037040 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.257+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a7c30 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.258+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4069ac0 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.260+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6ca4043020 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.262+0000 7f6cab400640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 0x7f6c980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.263+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f6ca40f0760 con 0x7f6cb4100c90 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.263+0000 7f6cb2000640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 0x7f6c980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.264+0000 7f6cb2000640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 0x7f6c980a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6cb410f540 tx=0x7f6ca0005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:26.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.267+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ca404a020 con 0x7f6cb4100c90 2024-04-26T22:13:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.477+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cb41a7e00 con 0x7f6cb4100c90 2024-04-26T22:13:26.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.479+0000 7f6cab400640 1 -- 172.21.15.12:0/3617579341 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25344 (secure 0 0 0) 0x7f6ca40bcab0 con 0x7f6cb4100c90 2024-04-26T22:13:26.482 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T22:13:26.482 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":105,"fsid":"d6d929b6-0416-11ef-bc93-c7b262605968","created":"2024-04-26T21:50:26.382191+0000","modified":"2024-04-26T22:13:16.884039+0000","last_up_change":"2024-04-26T21:58:45.344924+0000","last_in_change":"2024-04-26T21:58:25.416336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:54:29.133630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:59:41.200542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:59:42.493543+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T21:59:46.121636+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"26c2f4bf-8ef4-4f89-9dc8-b00f6e556980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6803","nonce":1061847783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6805","nonce":1061847783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6809","nonce":1061847783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1061847783},{"type":"v1","addr":"172.21.15.12:6807","nonce":1061847783}]},"public_addr":"172.21.15.12:6803/1061847783","cluster_addr":"172.21.15.12:6805/1061847783","heartbeat_back_addr":"172.21.15.12:6809/1061847783","heartbeat_front_addr":"172.21.15.12:6807/1061847783","state":["exists","up"]},{"osd":1,"uuid":"f493e866-faee-4d6e-a462-c582971cf31b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6801","nonce":828320390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6803","nonce":828320390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6807","nonce":828320390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":828320390},{"type":"v1","addr":"172.21.15.77:6805","nonce":828320390}]},"public_addr":"172.21.15.77:6801/828320390","cluster_addr":"172.21.15.77:6803/828320390","heartbeat_back_addr":"172.21.15.77:6807/828320390","heartbeat_front_addr":"172.21.15.77:6805/828320390","state":["exists","up"]},{"osd":2,"uuid":"1a38e236-f013-4ab3-9310-272f5ec1b653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6801","nonce":3122867729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6803","nonce":3122867729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6807","nonce":3122867729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3122867729},{"type":"v1","addr":"172.21.15.154:6805","nonce":3122867729}]},"public_addr":"172.21.15.154:6801/3122867729","cluster_addr":"172.21.15.154:6803/3122867729","heartbeat_back_addr":"172.21.15.154:6807/3122867729","heartbeat_front_addr":"172.21.15.154:6805/3122867729","state":["exists","up"]},{"osd":3,"uuid":"d44a553b-4160-4233-b691-acd3104b7eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6811","nonce":1602812044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6813","nonce":1602812044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6817","nonce":1602812044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1602812044},{"type":"v1","addr":"172.21.15.12:6815","nonce":1602812044}]},"public_addr":"172.21.15.12:6811/1602812044","cluster_addr":"172.21.15.12:6813/1602812044","heartbeat_back_addr":"172.21.15.12:6817/1602812044","heartbeat_front_addr":"172.21.15.12:6815/1602812044","state":["exists","up"]},{"osd":4,"uuid":"13cc095e-3560-47a1-bca1-4dc3cbbc8198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6809","nonce":482924396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6811","nonce":482924396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6815","nonce":482924396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":482924396},{"type":"v1","addr":"172.21.15.77:6813","nonce":482924396}]},"public_addr":"172.21.15.77:6809/482924396","cluster_addr":"172.21.15.77:6811/482924396","heartbeat_back_addr":"172.21.15.77:6815/482924396","heartbeat_front_addr":"172.21.15.77:6813/482924396","state":["exists","up"]},{"osd":5,"uuid":"38bc196f-d165-47ad-815e-da841218d3a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6809","nonce":1747631146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6811","nonce":1747631146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6815","nonce":1747631146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1747631146},{"type":"v1","addr":"172.21.15.154:6813","nonce":1747631146}]},"public_addr":"172.21.15.154:6809/1747631146","cluster_addr":"172.21.15.154:6811/1747631146","heartbeat_back_addr":"172.21.15.154:6815/1747631146","heartbeat_front_addr":"172.21.15.154:6813/1747631146","state":["exists","up"]},{"osd":6,"uuid":"b5475e1d-8f75-4b95-b8d5-45916e0ed62f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6819","nonce":3229053639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6821","nonce":3229053639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6825","nonce":3229053639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3229053639},{"type":"v1","addr":"172.21.15.12:6823","nonce":3229053639}]},"public_addr":"172.21.15.12:6819/3229053639","cluster_addr":"172.21.15.12:6821/3229053639","heartbeat_back_addr":"172.21.15.12:6825/3229053639","heartbeat_front_addr":"172.21.15.12:6823/3229053639","state":["exists","up"]},{"osd":7,"uuid":"00d199a4-d87f-4184-b154-13d6f3dd67e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6817","nonce":1953584019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6819","nonce":1953584019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6823","nonce":1953584019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1953584019},{"type":"v1","addr":"172.21.15.77:6821","nonce":1953584019}]},"public_addr":"172.21.15.77:6817/1953584019","cluster_addr":"172.21.15.77:6819/1953584019","heartbeat_back_addr":"172.21.15.77:6823/1953584019","heartbeat_front_addr":"172.21.15.77:6821/1953584019","state":["exists","up"]},{"osd":8,"uuid":"03a5cda5-c502-4af5-88e3-3196bf23380d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6817","nonce":165436981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6819","nonce":165436981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6823","nonce":165436981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":165436981},{"type":"v1","addr":"172.21.15.154:6821","nonce":165436981}]},"public_addr":"172.21.15.154:6817/165436981","cluster_addr":"172.21.15.154:6819/165436981","heartbeat_back_addr":"172.21.15.154:6823/165436981","heartbeat_front_addr":"172.21.15.154:6821/165436981","state":["exists","up"]},{"osd":9,"uuid":"70375e11-a2c2-4512-9403-6cc31fb3409c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6827","nonce":1774118135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6829","nonce":1774118135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6833","nonce":1774118135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1774118135},{"type":"v1","addr":"172.21.15.12:6831","nonce":1774118135}]},"public_addr":"172.21.15.12:6827/1774118135","cluster_addr":"172.21.15.12:6829/1774118135","heartbeat_back_addr":"172.21.15.12:6833/1774118135","heartbeat_front_addr":"172.21.15.12:6831/1774118135","state":["exists","up"]},{"osd":10,"uuid":"1a526fb3-2f0b-4161-9b40-9ba2774e847a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6825","nonce":1999551212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6827","nonce":1999551212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6831","nonce":1999551212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":1999551212},{"type":"v1","addr":"172.21.15.77:6829","nonce":1999551212}]},"public_addr":"172.21.15.77:6825/1999551212","cluster_addr":"172.21.15.77:6827/1999551212","heartbeat_back_addr":"172.21.15.77:6831/1999551212","heartbeat_front_addr":"172.21.15.77:6829/1999551212","state":["exists","up"]},{"osd":11,"uuid":"bfad9b30-8b56-4b29-9722-cf3de267af70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6825","nonce":3735995380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6827","nonce":3735995380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6831","nonce":3735995380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3735995380},{"type":"v1","addr":"172.21.15.154:6829","nonce":3735995380}]},"public_addr":"172.21.15.154:6825/3735995380","cluster_addr":"172.21.15.154:6827/3735995380","heartbeat_back_addr":"172.21.15.154:6831/3735995380","heartbeat_front_addr":"172.21.15.154:6829/3735995380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:53:36.901647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:01.873989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:25.741458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:54:59.617299+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:25.416862+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:55:49.887657+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:25.418230+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:56:51.430210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:16.513878+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:57:51.110689+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:17.641240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:58:43.379853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/1038496765":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6801/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/600788188":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6800/1377681366":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/1310858189":"2024-04-27T21:51:07.889475+0000","172.21.15.12:6801/3660687894":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/36130226":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/3243088974":"2024-04-27T21:51:07.889475+0000","172.21.15.12:0/3387900079":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/53237046":"2024-04-27T22:02:35.907627+0000","172.21.15.12:0/505816482":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2638555806":"2024-04-27T21:51:42.978579+0000","172.21.15.12:6801/2513368485":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/4197187136":"2024-04-27T21:51:42.978579+0000","172.21.15.12:0/2706265351":"2024-04-27T22:02:35.907627+0000","172.21.15.12:6800/3660687894":"2024-04-27T22:02:35.907627+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:13:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 msgr2=0x7f6c980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 0x7f6c980a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6cb410f540 tx=0x7f6ca0005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 msgr2=0x7f6cb419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009980 tx=0x7f6ca4032dc0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 shutdown_connections 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f6c980a3da0 0x7f6c980a6260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6cb4104510 0x7f6cb419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cb4101630 0x7f6cb419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 --2- 172.21.15.12:0/3617579341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cb4100c90 0x7f6cb419b0c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 >> 172.21.15.12:0/3617579341 conn(0x7f6cb4077c20 msgr2=0x7f6cb40ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.482+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 shutdown_connections 2024-04-26T22:13:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.483+0000 7f6cb3e00640 1 -- 172.21.15.12:0/3617579341 wait complete. 2024-04-26T22:13:26.504 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T22:13:26.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:26 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/2658668376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:26 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3305042005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.724+0000 7f10fa600640 1 -- 172.21.15.12:0/767369045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f4104900 msgr2=0x7f10f4104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.724+0000 7f10fa600640 1 --2- 172.21.15.12:0/767369045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f4104900 0x7f10f4104d80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009930 tx=0x7f10e002a440 comp rx=0 tx=0).stop 2024-04-26T22:13:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 -- 172.21.15.12:0/767369045 shutdown_connections 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 --2- 172.21.15.12:0/767369045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 0x7f10f41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 --2- 172.21.15.12:0/767369045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f4104900 0x7f10f4104d80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 --2- 172.21.15.12:0/767369045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f410a910 0x7f10f410acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 -- 172.21.15.12:0/767369045 >> 172.21.15.12:0/767369045 conn(0x7f10f41003d0 msgr2=0x7f10f41027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 -- 172.21.15.12:0/767369045 shutdown_connections 2024-04-26T22:13:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.725+0000 7f10fa600640 1 -- 172.21.15.12:0/767369045 wait complete. 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 Processor -- start 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 -- start start 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 0x7f10f419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4114c20 con 0x7f10f410a910 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4114aa0 con 0x7f10f41052c0 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.726+0000 7f10fa600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4114da0 con 0x7f10f4104900 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43014/0 (socket says 172.21.15.12:43014) 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45374/0 (socket says 172.21.15.12:45374) 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9200640 1 -- 172.21.15.12:0/3675833465 learned_addr learned my addr 172.21.15.12:0/3675833465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f3e00640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 0x7f10f419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 -- 172.21.15.12:0/3675833465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 msgr2=0x7f10f419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 -- 172.21.15.12:0/3675833465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 msgr2=0x7f10f419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 0x7f10f419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9c00640 1 -- 172.21.15.12:0/3675833465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f41a0120 con 0x7f10f410a910 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.727+0000 7f10f9200640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10f9c00640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00e970 tx=0x7f10ec00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00ccc0 con 0x7f10f410a910 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f41a0410 con 0x7f10f410a910 2024-04-26T22:13:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec005020 con 0x7f10f410a910 2024-04-26T22:13:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004a00 con 0x7f10f410a910 2024-04-26T22:13:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.728+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41a7ca0 con 0x7f10f410a910 2024-04-26T22:13:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.729+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f4069ac0 con 0x7f10f410a910 2024-04-26T22:13:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.731+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f10ec004ba0 con 0x7f10f410a910 2024-04-26T22:13:26.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:26 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/2658668376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:26 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3305042005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.733+0000 7f10f1600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 0x7f10dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.734+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9547+0+0 (secure 0 0 0) 0x7f10ec0d0b50 con 0x7f10f410a910 2024-04-26T22:13:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.734+0000 7f10f9200640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 0x7f10dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.735+0000 7f10f9200640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 0x7f10dc0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10e80049d0 tx=0x7f10e8009500 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.737+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f10ec09cdf0 con 0x7f10f410a910 2024-04-26T22:13:26.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:26 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/2658668376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:26 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3305042005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:13:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.987+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f10f410c900 con 0x7f10f410a910 2024-04-26T22:13:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.993+0000 7f10f1600640 1 -- 172.21.15.12:0/3675833465 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f10ec09c6f0 con 0x7f10f410a910 2024-04-26T22:13:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:13:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.995+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 msgr2=0x7f10dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.995+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 0x7f10dc0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10e80049d0 tx=0x7f10e8009500 comp rx=0 tx=0).stop 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.996+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 msgr2=0x7f10f419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.996+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00e970 tx=0x7f10ec00ee30 comp rx=0 tx=0).stop 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 shutdown_connections 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f10dc0a3d20 0x7f10dc0a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10f410a910 0x7f10f419f9f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10f41052c0 0x7f10f419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 --2- 172.21.15.12:0/3675833465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f4104900 0x7f10f419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.997+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 >> 172.21.15.12:0/3675833465 conn(0x7f10f41003d0 msgr2=0x7f10f4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.998+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 shutdown_connections 2024-04-26T22:13:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:26.999+0000 7f10fa600640 1 -- 172.21.15.12:0/3675833465 wait complete. 2024-04-26T22:13:27.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 -- 172.21.15.12:0/4236240020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801027e0 msgr2=0x7f2480102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 --2- 172.21.15.12:0/4236240020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801027e0 0x7f2480102c60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f246c009930 tx=0x7f246c02a420 comp rx=0 tx=0).stop 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 -- 172.21.15.12:0/4236240020 shutdown_connections 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 --2- 172.21.15.12:0/4236240020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801031a0 0x7f248010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 --2- 172.21.15.12:0/4236240020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801027e0 0x7f2480102c60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 --2- 172.21.15.12:0/4236240020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f2480108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 -- 172.21.15.12:0/4236240020 >> 172.21.15.12:0/4236240020 conn(0x7f24800fe2d0 msgr2=0x7f24801006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 -- 172.21.15.12:0/4236240020 shutdown_connections 2024-04-26T22:13:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.210+0000 7f247fe00640 1 -- 172.21.15.12:0/4236240020 wait complete. 2024-04-26T22:13:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.211+0000 7f247fe00640 1 Processor -- start 2024-04-26T22:13:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.211+0000 7f247fe00640 1 -- start start 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 0x7f2480074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f24800756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112ab0 con 0x7f24801087f0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112930 con 0x7f24801027e0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112c30 con 0x7f24801031a0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247ea00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 0x7f2480074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:43056/0 (socket says 172.21.15.12:43056) 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247e000640 1 -- 172.21.15.12:0/253207330 learned_addr learned my addr 172.21.15.12:0/253207330 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.212+0000 7f247f400640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f24800756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 -- 172.21.15.12:0/253207330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 msgr2=0x7f2480074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 0x7f2480074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 -- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 msgr2=0x7f24800756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f24800756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 -- 172.21.15.12:0/253207330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24801a7510 con 0x7f24801031a0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247ea00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 0x7f2480074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247f400640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f24800756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247e000640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f246c005940 tx=0x7f246c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c03f030 con 0x7f24801031a0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801a77a0 con 0x7f24801031a0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246c0044c0 con 0x7f24801031a0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.213+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c035070 con 0x7f24801031a0 2024-04-26T22:13:27.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.214+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801a7c90 con 0x7f24801031a0 2024-04-26T22:13:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.214+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2480069ac0 con 0x7f24801031a0 2024-04-26T22:13:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.217+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f246c042030 con 0x7f24801031a0 2024-04-26T22:13:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.218+0000 7f2477400640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.218+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9545+0+0 (secure 0 0 0) 0x7f246c0f12b0 con 0x7f24801031a0 2024-04-26T22:13:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.219+0000 7f247ea00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:27.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.219+0000 7f247ea00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 0x7f24640a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24700095f0 tx=0x7f2470005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.221+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f246c0bd4f0 con 0x7f24801031a0 2024-04-26T22:13:27.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a[28180]: 2024-04-26T22:13:26.989+0000 7fd2d0200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:13:27.323 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: ignoring --setuser ceph since I am not root 2024-04-26T22:13:27.323 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: ignoring --setgroup ceph since I am not root 2024-04-26T22:13:27.324 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mds[69314]: -- 172.21.15.12:0/3612058691 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581a6ca5a40 con 0x5581a6c79000 2024-04-26T22:13:27.324 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: starting mds.g at 2024-04-26T22:13:27.381 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: ignoring --setuser ceph since I am not root 2024-04-26T22:13:27.381 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: ignoring --setgroup ceph since I am not root 2024-04-26T22:13:27.381 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mds[62278]: -- 172.21.15.77:0/2342643122 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef7cc4da40 con 0x55ef7cc21000 2024-04-26T22:13:27.381 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: starting mds.b at 2024-04-26T22:13:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:27.472+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f248010ad70 con 0x7f24801031a0 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3617579341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:13:27.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T22:13:27.642 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:27 smithi154 ceph-mon[32578]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T22:13:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3617579341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:27.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:13:27.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:13:27.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:13:27.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:13:27.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T22:13:27.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:27 smithi077 ceph-mon[32280]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: pgmap v338: 129 pgs: 129 active+clean; 2.7 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3617579341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3675833465' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T22:13:27.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:27 smithi012 ceph-mon[28184]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T22:13:28.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.000+0000 7f2477400640 1 -- 172.21.15.12:0/253207330 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f246c0bcdf0 con 0x7f24801031a0 2024-04-26T22:13:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.003+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 msgr2=0x7f24640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.003+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 0x7f24640a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24700095f0 tx=0x7f2470005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.003+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 msgr2=0x7f2480075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.003+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f246c005940 tx=0x7f246c0040c0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 shutdown_connections 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24801087f0 0x7f24800756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24801031a0 0x7f2480075170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 --2- 172.21.15.12:0/253207330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24801027e0 0x7f2480074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 >> 172.21.15.12:0/253207330 conn(0x7f24800fe2d0 msgr2=0x7f24800feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 shutdown_connections 2024-04-26T22:13:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.004+0000 7f247fe00640 1 -- 172.21.15.12:0/253207330 wait complete. 2024-04-26T22:13:28.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 -- 172.21.15.12:0/195727573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f138410a9e0 msgr2=0x7f138410adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 --2- 172.21.15.12:0/195727573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f138410a9e0 0x7f138410adc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1374009930 tx=0x7f137402a420 comp rx=0 tx=0).stop 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 -- 172.21.15.12:0/195727573 shutdown_connections 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 --2- 172.21.15.12:0/195727573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 0x7f13841118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 --2- 172.21.15.12:0/195727573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13841049d0 0x7f1384104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 --2- 172.21.15.12:0/195727573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f138410a9e0 0x7f138410adc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.246+0000 7f1383e00640 1 -- 172.21.15.12:0/195727573 >> 172.21.15.12:0/195727573 conn(0x7f1384100460 msgr2=0x7f1384102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.247+0000 7f1383e00640 1 -- 172.21.15.12:0/195727573 shutdown_connections 2024-04-26T22:13:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.247+0000 7f1383e00640 1 -- 172.21.15.12:0/195727573 wait complete. 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 Processor -- start 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 -- start start 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 0x7f138419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f138410a9e0 0x7f138419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114cf0 con 0x7f13841049d0 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114b70 con 0x7f138410a9e0 2024-04-26T22:13:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1383e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114e70 con 0x7f1384105390 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1382a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1382a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45424/0 (socket says 172.21.15.12:45424) 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.248+0000 7f1382a00640 1 -- 172.21.15.12:0/1479106206 learned_addr learned my addr 172.21.15.12:0/1479106206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382000640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 0x7f138419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1383400640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f138410a9e0 0x7f138419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 -- 172.21.15.12:0/1479106206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 msgr2=0x7f138419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 0x7f138419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 -- 172.21.15.12:0/1479106206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f138410a9e0 msgr2=0x7f138419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f138410a9e0 0x7f138419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 -- 172.21.15.12:0/1479106206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13841a0210 con 0x7f13841049d0 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.249+0000 7f1382a00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1374005ab0 tx=0x7f1374003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.250+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137403f070 con 0x7f13841049d0 2024-04-26T22:13:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.250+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1374031b50 con 0x7f13841049d0 2024-04-26T22:13:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.250+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374035030 con 0x7f13841049d0 2024-04-26T22:13:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.250+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13841a04a0 con 0x7f13841049d0 2024-04-26T22:13:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.250+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a7d90 con 0x7f13841049d0 2024-04-26T22:13:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.251+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1384069670 con 0x7f13841049d0 2024-04-26T22:13:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.253+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1374042070 con 0x7f13841049d0 2024-04-26T22:13:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.254+0000 7f137b400640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 0x7f13680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.255+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9545+0+0 (secure 0 0 0) 0x7f13740f0dd0 con 0x7f13841049d0 2024-04-26T22:13:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.255+0000 7f1382000640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 0x7f13680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.256+0000 7f1382000640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 0x7f13680a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13700095f0 tx=0x7f1370005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.260+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f13740bc9c0 con 0x7f13841049d0 2024-04-26T22:13:28.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:28.466+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f138419c570 con 0x7f13841049d0 2024-04-26T22:13:28.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/253207330' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: Cluster is now healthy 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.77:6838/3762944539,v1:172.21.15.77:6839/3762944539] up:boot 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: mds.? [v2:172.21.15.12:6838/2052097565,v1:172.21.15.12:6839/2052097565] up:boot 2024-04-26T22:13:28.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:28 smithi154 ceph-mon[32578]: fsmap 12 up:standby 2024-04-26T22:13:28.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/253207330' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: Cluster is now healthy 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.77:6838/3762944539,v1:172.21.15.77:6839/3762944539] up:boot 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: mds.? [v2:172.21.15.12:6838/2052097565,v1:172.21.15.12:6839/2052097565] up:boot 2024-04-26T22:13:28.733 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:28 smithi077 ceph-mon[32280]: fsmap 12 up:standby 2024-04-26T22:13:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/253207330' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:13:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:13:28.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: Cluster is now healthy 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: from='mgr.25318 172.21.15.12:0/2919874765' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.77:6838/3762944539,v1:172.21.15.77:6839/3762944539] up:boot 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: mds.? [v2:172.21.15.12:6838/2052097565,v1:172.21.15.12:6839/2052097565] up:boot 2024-04-26T22:13:28.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:28 smithi012 ceph-mon[28184]: fsmap 12 up:standby 2024-04-26T22:13:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.398+0000 7f137b400640 1 -- 172.21.15.12:0/1479106206 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f13740c0cc0 con 0x7f13841049d0 2024-04-26T22:13:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-04-26T22:13:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.402+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 msgr2=0x7f13680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.402+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 0x7f13680a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f13700095f0 tx=0x7f1370005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.402+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 msgr2=0x7f138419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.402+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1374005ab0 tx=0x7f1374003b90 comp rx=0 tx=0).stop 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.402+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 shutdown_connections 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.403+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f13680a3da0 0x7f13680a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.403+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f138410a9e0 0x7f138419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.403+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1384105390 0x7f138419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.403+0000 7f1383e00640 1 --2- 172.21.15.12:0/1479106206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13841049d0 0x7f138419b270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.403+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 >> 172.21.15.12:0/1479106206 conn(0x7f1384100460 msgr2=0x7f1384100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.404+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 shutdown_connections 2024-04-26T22:13:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.404+0000 7f1383e00640 1 -- 172.21.15.12:0/1479106206 wait complete. 2024-04-26T22:13:29.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T22:13:29.640 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:29 smithi154 ceph-mon[32578]: pgmap v340: 129 pgs: 129 active+clean; 2.7 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-04-26T22:13:29.641 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:29 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:29.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 -- 172.21.15.12:0/1213946221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4084074d60 msgr2=0x7f4084075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:29.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 --2- 172.21.15.12:0/1213946221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4084074d60 0x7f4084075140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4074009ca0 tx=0x7f407402a950 comp rx=0 tx=0).stop 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 -- 172.21.15.12:0/1213946221 shutdown_connections 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 --2- 172.21.15.12:0/1213946221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f408410f010 0x7f4084111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 --2- 172.21.15.12:0/1213946221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084075680 0x7f408410ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 --2- 172.21.15.12:0/1213946221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4084074d60 0x7f4084075140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 -- 172.21.15.12:0/1213946221 >> 172.21.15.12:0/1213946221 conn(0x7f40840fe300 msgr2=0x7f4084100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.644+0000 7f4089a00640 1 -- 172.21.15.12:0/1213946221 shutdown_connections 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.645+0000 7f4089a00640 1 -- 172.21.15.12:0/1213946221 wait complete. 2024-04-26T22:13:29.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 Processor -- start 2024-04-26T22:13:29.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 -- start start 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 0x7f408419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4084075680 0x7f408419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4084114cb0 con 0x7f4084074d60 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4084114b30 con 0x7f4084075680 2024-04-26T22:13:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.646+0000 7f4089a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4084114e30 con 0x7f408410f010 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4083e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 0x7f408419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.12:37684/0 (socket says 172.21.15.12:37684) 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 -- 172.21.15.12:0/1338481093 learned_addr learned my addr 172.21.15.12:0/1338481093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f407b400640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4084075680 0x7f408419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 -- 172.21.15.12:0/1338481093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4084075680 msgr2=0x7f408419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4084075680 0x7f408419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 -- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 msgr2=0x7f408419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 0x7f408419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4089000640 1 -- 172.21.15.12:0/1338481093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40841a01f0 con 0x7f408410f010 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.647+0000 7f4083e00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 0x7f408419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4089000640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e970 tx=0x7f407c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c00ccc0 con 0x7f408410f010 2024-04-26T22:13:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c00ce60 con 0x7f408410f010 2024-04-26T22:13:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40841a04e0 con 0x7f408410f010 2024-04-26T22:13:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c010430 con 0x7f408410f010 2024-04-26T22:13:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.648+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40841a7d70 con 0x7f408410f010 2024-04-26T22:13:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.649+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4084069670 con 0x7f408410f010 2024-04-26T22:13:29.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.652+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f407c004f90 con 0x7f408410f010 2024-04-26T22:13:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.653+0000 7f4081600640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 0x7f40680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.654+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8983+0+0 (secure 0 0 0) 0x7f407c0d0ce0 con 0x7f408410f010 2024-04-26T22:13:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.654+0000 7f4083e00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 0x7f40680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.655+0000 7f4083e00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 0x7f40680a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f407402af40 tx=0x7f4074005d80 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.658+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f407c09d150 con 0x7f408410f010 2024-04-26T22:13:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:29 smithi077 ceph-mon[32280]: pgmap v340: 129 pgs: 129 active+clean; 2.7 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-04-26T22:13:29.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:29 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:29 smithi012 ceph-mon[28184]: pgmap v340: 129 pgs: 129 active+clean; 2.7 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-04-26T22:13:29.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:29 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:29.865+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f408410a1c0 con 0x7f408410f010 2024-04-26T22:13:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.406+0000 7f4081600640 1 -- 172.21.15.12:0/1338481093 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7f407c09ca50 con 0x7f408410f010 2024-04-26T22:13:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-04-26T22:13:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 msgr2=0x7f40680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 0x7f40680a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f407402af40 tx=0x7f4074005d80 comp rx=0 tx=0).stop 2024-04-26T22:13:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 msgr2=0x7f408419fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e970 tx=0x7f407c00ed70 comp rx=0 tx=0).stop 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 shutdown_connections 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7f40680a3da0 0x7f40680a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f408410f010 0x7f408419fac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4084075680 0x7f408419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 --2- 172.21.15.12:0/1338481093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4084074d60 0x7f408419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.412+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 >> 172.21.15.12:0/1338481093 conn(0x7f40840fe300 msgr2=0x7f40840ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.413+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 shutdown_connections 2024-04-26T22:13:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.413+0000 7f4089a00640 1 -- 172.21.15.12:0/1338481093 wait complete. 2024-04-26T22:13:30.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T22:13:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 -- 172.21.15.12:0/3219920610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8105490 msgr2=0x7fedc81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 --2- 172.21.15.12:0/3219920610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8105490 0x7fedc81119a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009cd0 tx=0x7fedc002a8f0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 -- 172.21.15.12:0/3219920610 shutdown_connections 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 --2- 172.21.15.12:0/3219920610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8105490 0x7fedc81119a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 --2- 172.21.15.12:0/3219920610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8104ad0 0x7fedc8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 --2- 172.21.15.12:0/3219920610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 -- 172.21.15.12:0/3219920610 >> 172.21.15.12:0/3219920610 conn(0x7fedc8100660 msgr2=0x7fedc8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 -- 172.21.15.12:0/3219920610 shutdown_connections 2024-04-26T22:13:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.658+0000 7fedce000640 1 -- 172.21.15.12:0/3219920610 wait complete. 2024-04-26T22:13:30.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.659+0000 7fedce000640 1 Processor -- start 2024-04-26T22:13:30.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.659+0000 7fedce000640 1 -- start start 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8104ad0 0x7fedc819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8114df0 con 0x7fedc810ab70 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8114c70 con 0x7fedc8105490 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedce000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8114f70 con 0x7fedc8104ad0 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.12:47984/0 (socket says 172.21.15.12:47984) 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51318/0 (socket says 172.21.15.12:51318) 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 -- 172.21.15.12:0/3114425289 learned_addr learned my addr 172.21.15.12:0/3114425289 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc7400640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8104ad0 0x7fedc819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 -- 172.21.15.12:0/3114425289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8104ad0 msgr2=0x7fedc819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8104ad0 0x7fedc819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 -- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 msgr2=0x7fedc819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.660+0000 7fedc6a00640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedc6a00640 1 -- 172.21.15.12:0/3114425289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc81a02c0 con 0x7fedc8105490 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedc7e00640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:13:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedc6a00640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedb400e9f0 tx=0x7fedb400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb400cd80 con 0x7fedc8105490 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb40054d0 con 0x7fedc8105490 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc81a05b0 con 0x7fedc8105490 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4010690 con 0x7fedc8105490 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.661+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc81a7e40 con 0x7fedc8105490 2024-04-26T22:13:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.662+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc8069ba0 con 0x7fedc8105490 2024-04-26T22:13:30.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.665+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fedb4004e30 con 0x7fedc8105490 2024-04-26T22:13:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.667+0000 7fedbfe00640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 0x7fedac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:13:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.667+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8453+0+0 (secure 0 0 0) 0x7fedb40d0e10 con 0x7fedc8105490 2024-04-26T22:13:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.667+0000 7fedc7400640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 0x7fedac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:13:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.668+0000 7fedc7400640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 0x7fedac0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009b50 tx=0x7fedb8005f90 comp rx=0 tx=0).ready entity=mgr.25318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:13:30.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.668+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fedb4054060 con 0x7fedc8105490 2024-04-26T22:13:30.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:30 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:30.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:30 smithi077 ceph-mon[32280]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T22:13:30.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:30 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/1338481093' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:30.734 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:30 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:30 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:30 smithi012 ceph-mon[28184]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T22:13:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:30 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/1338481093' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:30.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:30 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:30.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:30.877+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fedc819c620 con 0x7fedc8105490 2024-04-26T22:13:30.890 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:30 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1479106206' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:30 smithi154 ceph-mon[32578]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T22:13:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:30 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/1338481093' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:30.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:30 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.435+0000 7fedbfe00640 1 -- 172.21.15.12:0/3114425289 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) ==== 178+0+0 (secure 0 0 0) 0x7fedc819c620 con 0x7fedc8105490 2024-04-26T22:13:31.438 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 msgr2=0x7fedac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 0x7fedac0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009b50 tx=0x7fedb8005f90 comp rx=0 tx=0).stop 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 msgr2=0x7fedc819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedb400e9f0 tx=0x7fedb400edf0 comp rx=0 tx=0).stop 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 shutdown_connections 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:6800/2662094264,v1:172.21.15.12:6801/2662094264] conn(0x7fedac0a3d20 0x7fedac0a61e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:31.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedc810ab70 0x7fedc819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc8105490 0x7fedc819b810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 --2- 172.21.15.12:0/3114425289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedc8104ad0 0x7fedc819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:13:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.442+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 >> 172.21.15.12:0/3114425289 conn(0x7fedc8100660 msgr2=0x7fedc8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:13:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.443+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 shutdown_connections 2024-04-26T22:13:31.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T22:13:31.443+0000 7fedce000640 1 -- 172.21.15.12:0/3114425289 wait complete. 2024-04-26T22:13:31.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:13:31.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:13:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:13:31.621 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T22:13:31.621 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:13:31.672 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:13:31.688 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:13:31.715 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T22:13:31.716 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T22:13:31.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a 2024-04-26T22:13:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:31 smithi077 ceph-mon[32280]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:13:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:31 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:31 smithi077 ceph-mon[32280]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T22:13:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:31 smithi077 ceph-mon[32280]: from='client.? 172.21.15.12:0/3114425289' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.732 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:31 smithi077 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 ceph-mon[28184]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:13:31.803 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:31.803 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 ceph-mon[28184]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T22:13:31.803 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 ceph-mon[28184]: from='client.? 172.21.15.12:0/3114425289' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.803 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 ceph-mon[28184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:31 smithi154 ceph-mon[32578]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T22:13:31.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:31 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:13:31.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:31 smithi154 ceph-mon[32578]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T22:13:31.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:31 smithi154 ceph-mon[32578]: from='client.? 172.21.15.12:0/3114425289' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:31.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:31 smithi154 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:13:32.074 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:31 smithi012 systemd[1]: Stopping Ceph mon.a for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a[28180]: 2024-04-26T22:13:32.378+0000 7fd2d7000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a[28180]: 2024-04-26T22:13:32.378+0000 7fd2d7000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T22:13:32.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:32 smithi012 podman[88324]: 2024-04-26 22:13:32.564318948 +0000 UTC m=+0.385968536 container died 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T22:13:33.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:33 smithi012 podman[88324]: 2024-04-26 22:13:33.120545108 +0000 UTC m=+0.942194717 container cleanup 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T22:13:33.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:33 smithi012 bash[88324]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a 2024-04-26T22:13:33.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 26 22:13:33 smithi012 podman[88337]: 2024-04-26 22:13:33.489270594 +0000 UTC m=+0.907501919 container remove 057f9fa44791b200b139089b3bf878a1143df9dcdc156a2c69196c48ad5b4b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T22:13:34.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T22:13:34.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:34.300 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T22:13:34.301 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T22:13:34.301 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.b 2024-04-26T22:13:34.603 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:34 smithi077 systemd[1]: Stopping Ceph mon.b for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:34.982 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-b[32276]: 2024-04-26T22:13:34.603+0000 7f2825042700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:34.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-b[32276]: 2024-04-26T22:13:34.603+0000 7f2825042700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:13:34.983 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:34 smithi077 podman[67670]: 2024-04-26 22:13:34.715823289 +0000 UTC m=+0.226650554 container died d7a1f61a5efe57f34b8ab75c0ac02a9bb581f1c0f3c0cd63f4892dddcd2e0001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T22:13:35.371 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:35 smithi077 podman[67670]: 2024-04-26 22:13:35.103628884 +0000 UTC m=+0.614456163 container cleanup d7a1f61a5efe57f34b8ab75c0ac02a9bb581f1c0f3c0cd63f4892dddcd2e0001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T22:13:35.371 INFO:journalctl@ceph.mon.b.smithi077.stdout:Apr 26 22:13:35 smithi077 bash[67670]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-b 2024-04-26T22:13:35.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T22:13:35.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:35.525 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T22:13:35.525 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T22:13:35.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.c 2024-04-26T22:13:35.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:35 smithi154 systemd[1]: Stopping Ceph mon.c for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c[32574]: 2024-04-26T22:13:35.840+0000 7f7c97ea6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c[32574]: 2024-04-26T22:13:35.840+0000 7f7c97ea6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:13:36.141 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:35 smithi154 podman[68798]: 2024-04-26 22:13:35.983890589 +0000 UTC m=+0.263786701 container died e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:13:36.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 podman[68798]: 2024-04-26 22:13:36.375162286 +0000 UTC m=+0.655058407 container cleanup e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:13:36.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 bash[68798]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c 2024-04-26T22:13:36.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 podman[68812]: 2024-04-26 22:13:36.593795787 +0000 UTC m=+0.610829317 container remove e63615a5e89d92c88b3b563a8bdadf9cce2eff7f4a9e927806864c52e9fa66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T22:13:36.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-26T22:13:36.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 systemd[1]: Stopped Ceph mon.c for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T22:13:36.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 26 22:13:36 smithi154 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.c.service: Consumed 13.540s CPU time. 2024-04-26T22:13:36.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T22:13:36.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:36.898 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T22:13:36.898 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T22:13:36.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x 2024-04-26T22:13:37.324 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:37 smithi012 systemd[1]: Stopping Ceph mgr.x for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:38.073 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x[28423]: 2024-04-26T22:13:37.598+0000 7fde7ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:38.074 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:37 smithi012 podman[88444]: 2024-04-26 22:13:37.704421479 +0000 UTC m=+0.336154537 container died 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:13:38.487 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:38 smithi012 podman[88444]: 2024-04-26 22:13:38.259929794 +0000 UTC m=+0.891662859 container cleanup 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T22:13:38.487 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:38 smithi012 bash[88444]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x 2024-04-26T22:13:38.823 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 26 22:13:38 smithi012 podman[88457]: 2024-04-26 22:13:38.486509833 +0000 UTC m=+0.769936969 container remove 0bfdf66e9f03c82b820f6331896a5b6bb4e598d817e0afb7e64ca2764b90dd4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:13:39.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T22:13:39.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:39.131 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T22:13:39.132 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T22:13:39.132 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y 2024-04-26T22:13:39.453 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:39 smithi077 systemd[1]: Stopping Ceph mgr.y for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:39.732 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:39 smithi077 podman[67790]: 2024-04-26 22:13:39.454140028 +0000 UTC m=+0.134117726 container died 23cae74151fb209275051c1c693713cf10c247d136fc563214edcbdf1a5d458b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T22:13:40.110 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:39 smithi077 podman[67790]: 2024-04-26 22:13:39.884012631 +0000 UTC m=+0.563990337 container cleanup 23cae74151fb209275051c1c693713cf10c247d136fc563214edcbdf1a5d458b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:13:40.110 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:39 smithi077 bash[67790]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y 2024-04-26T22:13:40.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T22:13:40.373 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:40 smithi077 podman[67805]: 2024-04-26 22:13:40.110618069 +0000 UTC m=+0.652621181 container remove 23cae74151fb209275051c1c693713cf10c247d136fc563214edcbdf1a5d458b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T22:13:40.373 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:40 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-26T22:13:40.373 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:40 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-26T22:13:40.373 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:40 smithi077 systemd[1]: Stopped Ceph mgr.y for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T22:13:40.373 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Apr 26 22:13:40 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.451s CPU time. 2024-04-26T22:13:40.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:40.385 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T22:13:40.385 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T22:13:40.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.z 2024-04-26T22:13:40.755 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:13:40 smithi154 systemd[1]: Stopping Ceph mgr.z for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:41.131 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:13:40 smithi154 podman[68920]: 2024-04-26 22:13:40.755828822 +0000 UTC m=+0.169468787 container died d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T22:13:41.391 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:13:41 smithi154 podman[68920]: 2024-04-26 22:13:41.250340966 +0000 UTC m=+0.663980940 container cleanup d1d8d36241ea9e2e311236a1b7625a7150fe6ff9c91b1529391ba50092c54a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T22:13:41.391 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 26 22:13:41 smithi154 bash[68920]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mgr-z 2024-04-26T22:13:41.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T22:13:41.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:41.708 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T22:13:41.708 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T22:13:41.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.0 2024-04-26T22:13:42.074 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:41 smithi012 systemd[1]: Stopping Ceph osd.0 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:42.573 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0[43289]: 2024-04-26T22:13:42.286+0000 7fd1b0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:42.573 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0[43289]: 2024-04-26T22:13:42.286+0000 7fd1b0c00700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-26T22:13:42.574 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0[43289]: 2024-04-26T22:13:42.286+0000 7fd1b0c00700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:13:48.212 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:47 smithi012 podman[88565]: 2024-04-26 22:13:47.914072782 +0000 UTC m=+5.762077308 container died 4086721f50760d01be3dd9937b41db879638edadc7b6f3dde5992e85fb3f2458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:13:48.573 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:48 smithi012 podman[88565]: 2024-04-26 22:13:48.339222246 +0000 UTC m=+6.187226795 container cleanup 4086721f50760d01be3dd9937b41db879638edadc7b6f3dde5992e85fb3f2458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:13:48.573 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:48 smithi012 bash[88565]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0 2024-04-26T22:13:49.073 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:48 smithi012 podman[88586]: 2024-04-26 22:13:48.774020842 +0000 UTC m=+0.845102354 container remove 4086721f50760d01be3dd9937b41db879638edadc7b6f3dde5992e85fb3f2458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T22:13:50.021 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:49 smithi012 podman[88679]: 2024-04-26 22:13:49.578750616 +0000 UTC m=+0.037851662 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:13:50.022 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:49 smithi012 podman[88679]: 2024-04-26 22:13:49.733639298 +0000 UTC m=+0.192740342 container create c858eb10e4b87dd69d8db34a327c7a48f42262cb67fba3ad9b88eb28630d88b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:13:50.323 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:50 smithi012 podman[88679]: 2024-04-26 22:13:50.021293008 +0000 UTC m=+0.480394092 container init c858eb10e4b87dd69d8db34a327c7a48f42262cb67fba3ad9b88eb28630d88b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:13:50.323 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:50 smithi012 podman[88679]: 2024-04-26 22:13:50.032858328 +0000 UTC m=+0.491959379 container start c858eb10e4b87dd69d8db34a327c7a48f42262cb67fba3ad9b88eb28630d88b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:13:50.323 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:50 smithi012 podman[88679]: 2024-04-26 22:13:50.136787296 +0000 UTC m=+0.595888321 container attach c858eb10e4b87dd69d8db34a327c7a48f42262cb67fba3ad9b88eb28630d88b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T22:13:50.823 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 26 22:13:50 smithi012 podman[88679]: 2024-04-26 22:13:50.503265138 +0000 UTC m=+0.962366212 container died c858eb10e4b87dd69d8db34a327c7a48f42262cb67fba3ad9b88eb28630d88b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T22:13:51.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T22:13:51.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:51.346 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T22:13:51.346 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T22:13:51.346 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.1 2024-04-26T22:13:51.638 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:51 smithi077 systemd[1]: Stopping Ceph osd.1 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:13:51.982 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1[36868]: 2024-04-26T22:13:51.638+0000 7fe567833700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:13:51.982 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1[36868]: 2024-04-26T22:13:51.638+0000 7fe567833700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-26T22:13:51.982 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1[36868]: 2024-04-26T22:13:51.638+0000 7fe567833700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:13:57.447 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:57 smithi077 podman[67929]: 2024-04-26 22:13:57.11125145 +0000 UTC m=+5.578488401 container died d73567eb1d715c844463912728f3d97d39bc8bfeee9f828f39eca6d95d7d6934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T22:13:57.697 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:57 smithi077 podman[67929]: 2024-04-26 22:13:57.447010444 +0000 UTC m=+5.914247408 container cleanup d73567eb1d715c844463912728f3d97d39bc8bfeee9f828f39eca6d95d7d6934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:13:57.698 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:57 smithi077 bash[67929]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1 2024-04-26T22:13:57.982 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:57 smithi077 podman[67946]: 2024-04-26 22:13:57.69822347 +0000 UTC m=+0.578715062 container remove d73567eb1d715c844463912728f3d97d39bc8bfeee9f828f39eca6d95d7d6934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T22:13:58.434 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.029420188 +0000 UTC m=+0.017761177 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:13:58.434 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.157510724 +0000 UTC m=+0.145851706 container create c590f304db4fee1a06db322b442954c47a66231d03cea6c409f65c7f6700afd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T22:13:58.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.4352178 +0000 UTC m=+0.423558778 container init c590f304db4fee1a06db322b442954c47a66231d03cea6c409f65c7f6700afd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T22:13:58.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.438817393 +0000 UTC m=+0.427158369 container start c590f304db4fee1a06db322b442954c47a66231d03cea6c409f65c7f6700afd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T22:13:58.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.543525088 +0000 UTC m=+0.531866072 container attach c590f304db4fee1a06db322b442954c47a66231d03cea6c409f65c7f6700afd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:13:58.982 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 22:13:58 smithi077 podman[68037]: 2024-04-26 22:13:58.731432301 +0000 UTC m=+0.719773274 container died c590f304db4fee1a06db322b442954c47a66231d03cea6c409f65c7f6700afd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:13:59.335 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T22:13:59.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:13:59.374 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T22:13:59.374 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T22:13:59.374 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.2 2024-04-26T22:13:59.676 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:13:59 smithi154 systemd[1]: Stopping Ceph osd.2 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:00.140 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:13:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2[37745]: 2024-04-26T22:13:59.676+0000 7fd3c36b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:00.141 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:13:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2[37745]: 2024-04-26T22:13:59.676+0000 7fd3c36b8700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-26T22:14:00.141 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:13:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2[37745]: 2024-04-26T22:13:59.676+0000 7fd3c36b8700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:05.386 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:05 smithi154 podman[69046]: 2024-04-26 22:14:05.086186038 +0000 UTC m=+5.525810017 container died 00162dbed153d0d84ce0bbd8d9ca96ee1fbd26e07cad9bb84e3be9ffd4f02baa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T22:14:05.640 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:05 smithi154 podman[69046]: 2024-04-26 22:14:05.477449605 +0000 UTC m=+5.917073586 container cleanup 00162dbed153d0d84ce0bbd8d9ca96ee1fbd26e07cad9bb84e3be9ffd4f02baa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:14:05.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:05 smithi154 bash[69046]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2 2024-04-26T22:14:06.098 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:05 smithi154 podman[69075]: 2024-04-26 22:14:05.662159993 +0000 UTC m=+0.577930959 container remove 00162dbed153d0d84ce0bbd8d9ca96ee1fbd26e07cad9bb84e3be9ffd4f02baa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T22:14:06.373 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.002160418 +0000 UTC m=+0.018026673 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:06.373 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.121764012 +0000 UTC m=+0.137630300 container create 4e6de2dd2987476d9c59244e3ec7db97a741a6077ccb0a6ce35256f9a4f277c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T22:14:06.640 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.37434509 +0000 UTC m=+0.390211335 container init 4e6de2dd2987476d9c59244e3ec7db97a741a6077ccb0a6ce35256f9a4f277c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:14:06.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.377643389 +0000 UTC m=+0.393509637 container start 4e6de2dd2987476d9c59244e3ec7db97a741a6077ccb0a6ce35256f9a4f277c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T22:14:06.641 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.474891732 +0000 UTC m=+0.490757987 container attach 4e6de2dd2987476d9c59244e3ec7db97a741a6077ccb0a6ce35256f9a4f277c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:14:06.994 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 26 22:14:06 smithi154 podman[69168]: 2024-04-26 22:14:06.670267781 +0000 UTC m=+0.686134026 container died 4e6de2dd2987476d9c59244e3ec7db97a741a6077ccb0a6ce35256f9a4f277c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:14:07.310 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T22:14:07.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:07.351 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T22:14:07.351 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T22:14:07.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.3 2024-04-26T22:14:07.824 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:07 smithi012 systemd[1]: Stopping Ceph osd.3 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:08.324 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3[49966]: 2024-04-26T22:14:07.967+0000 7fd708400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:08.324 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3[49966]: 2024-04-26T22:14:07.967+0000 7fd708400700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-26T22:14:08.324 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:07 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3[49966]: 2024-04-26T22:14:07.967+0000 7fd708400700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:08.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:08.206+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:09.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:08 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:08.973+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:09.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:09.111+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:09.205+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:10.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:10.002+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:10.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:10.072+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:10.207+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:11.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:10 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:10.977+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:11.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:11.086+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:11.201+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:11.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:10.981+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:12.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:12 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:12.013+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:12.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:11 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:11.808+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:12.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:12.234+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:12.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:12.043+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:12.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:12.024+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:13.044 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:12 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3[49966]: 2024-04-26T22:14:12.597+0000 7fd703c00700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.527306+0000 front 2024-04-26T22:13:44.528087+0000 (oldest deadline 2024-04-26T22:14:12.129172+0000) 2024-04-26T22:14:13.044 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:12 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:12.776+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:13.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:13.042+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:13.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:13.057+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:13.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:13.217+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:13.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:13.003+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:13.993 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:13.735+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:13.994 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:13 smithi012 podman[88796]: 2024-04-26 22:14:13.630245671 +0000 UTC m=+5.828346140 container died 531b0f14cc7585a3a516e98251fc4b2227dca78141fd7b34d150cf0b04f231b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T22:14:14.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:13 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:13.992+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:14.323 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:14 smithi012 podman[88796]: 2024-04-26 22:14:14.175736364 +0000 UTC m=+6.373836841 container cleanup 531b0f14cc7585a3a516e98251fc4b2227dca78141fd7b34d150cf0b04f231b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:14:14.323 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:14 smithi012 bash[88796]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3 2024-04-26T22:14:14.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:14.242+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:14.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:14.086+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:14.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:14 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:14.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:14.708 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:14 smithi012 podman[88814]: 2024-04-26 22:14:14.435755736 +0000 UTC m=+0.786644163 container remove 531b0f14cc7585a3a516e98251fc4b2227dca78141fd7b34d150cf0b04f231b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-26T22:14:14.961 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:14 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:14.707+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:15.309 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:14 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:14.960+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:15.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:15.060+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:15.274+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:15.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:15 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:15.035+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:15.573 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:15 smithi012 podman[88901]: 2024-04-26 22:14:15.217936403 +0000 UTC m=+0.039121731 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:15.573 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:15 smithi012 podman[88901]: 2024-04-26 22:14:15.353295433 +0000 UTC m=+0.174480806 container create c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T22:14:16.001 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:15 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:15.736+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:16.001 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:15 smithi012 podman[88901]: 2024-04-26 22:14:15.615513463 +0000 UTC m=+0.436698819 container init c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:16.001 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:15 smithi012 podman[88901]: 2024-04-26 22:14:15.62659553 +0000 UTC m=+0.447780886 container start c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T22:14:16.001 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:15 smithi012 podman[88901]: 2024-04-26 22:14:15.7810216 +0000 UTC m=+0.602206959 container attach c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-26T22:14:16.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:16 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:16.000+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:16.324 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:16 smithi012 podman[88901]: 2024-04-26 22:14:16.119128112 +0000 UTC m=+0.940313501 container died c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:14:16.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:16.076+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:16.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:16.251+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:16.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:16 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:16.083+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:16.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T22:14:17.018 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:16 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:16.700+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:17.018 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:16 smithi012 podman[88935]: 2024-04-26 22:14:16.901761034 +0000 UTC m=+0.764260122 container remove c16b5b1b0643a9c02e451c56d4bc4c28e3bcb2e5d0a54a71eee8b693a2b4426f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T22:14:17.018 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:16 smithi012 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-26T22:14:17.018 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:16 smithi012 systemd[1]: Stopped Ceph osd.3 for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T22:14:17.018 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 26 22:14:16 smithi012 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 16.258s CPU time. 2024-04-26T22:14:17.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:17.051 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T22:14:17.051 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T22:14:17.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.4 2024-04-26T22:14:17.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:17 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:17.034+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:17.358 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:17.069+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:17.358 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:17.269+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:17.366 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:17 smithi077 systemd[1]: Stopping Ceph osd.4 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:17.366 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:17 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:17.131+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:17.732 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:17 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:17.366+0000 7f9269445700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:17.732 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:17 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:17.366+0000 7f9269445700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-26T22:14:17.732 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:17 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:17.366+0000 7f9269445700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:17.993 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:17 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:17.708+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:18.139 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:17 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:17.888+0000 7f9265a39700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.379200+0000 front 2024-04-26T22:13:53.379344+0000 (oldest deadline 2024-04-26T22:14:17.479347+0000) 2024-04-26T22:14:18.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:17 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:17.993+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:18.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:18.084+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:18.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:18.084+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:18.237+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:18.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:18 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:18.139+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:18.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:18.550+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:18.955 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:18 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:18.840+0000 7f9265a39700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.379200+0000 front 2024-04-26T22:13:53.379344+0000 (oldest deadline 2024-04-26T22:14:17.479347+0000) 2024-04-26T22:14:19.028 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:18 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:18.742+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:19.028 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:18 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:18.742+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:19.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:18 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:18.955+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:19.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:19 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:19.177+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:19.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:19 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:19.177+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:19.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:19 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:19.027+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:19.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:19.038+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:19.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:19.038+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:19.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:19.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:19.578+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:20.056 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:19 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:19.710+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:20.056 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:19 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:19.710+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:20.226 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:19 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:19.856+0000 7f9265a39700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.379200+0000 front 2024-04-26T22:13:53.379344+0000 (oldest deadline 2024-04-26T22:14:17.479347+0000) 2024-04-26T22:14:20.226 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:19 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:19.960+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:20.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:20 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:20.056+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:20.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:20.262+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:20.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:20.032+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:20.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:20.032+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:20.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:20 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:20.227+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:20.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:20 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:20.227+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:20.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:20.616+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:21.060 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:20 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:20.721+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:21.060 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:20 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:20.721+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:21.228 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:20 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:20.842+0000 7f9265a39700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.379200+0000 front 2024-04-26T22:13:53.379344+0000 (oldest deadline 2024-04-26T22:14:17.479347+0000) 2024-04-26T22:14:21.228 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:21 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:21.003+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:21.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:21 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:21.060+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:21.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:21 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:21.060+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:21.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:21 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:21.272+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:21.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:21 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:21.057+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:21.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:21 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:21.057+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:21.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:21 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:21.227+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:21.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:21 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:21.227+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:21.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:21 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:21.623+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:22.038 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:21 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:21.682+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:22.039 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:21 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:21.682+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:22.189 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:21 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4[43817]: 2024-04-26T22:14:21.803+0000 7f9265a39700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.379200+0000 front 2024-04-26T22:13:53.379344+0000 (oldest deadline 2024-04-26T22:14:17.479347+0000) 2024-04-26T22:14:22.189 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:21 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:21.961+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:22.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:22 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:22.038+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:22.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:22 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:22.038+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:22.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:22 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:22.232+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:22.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:22 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:22.057+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:22.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:22 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:22.057+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:22.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:22 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:22.189+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:22.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:22 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:22.189+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:23.033 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:22 smithi077 podman[68157]: 2024-04-26 22:14:22.777776571 +0000 UTC m=+5.530174383 container died 377cc377708ad980c31e64a19a56dd4f0ac1f1d053a251054c16f92327a47eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T22:14:23.034 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:22 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:22.937+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:23.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:22 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:22.691+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:23.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:22 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:22.691+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:23.104 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:22 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:22.654+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:23.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:23 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:23.064+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:23.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:23 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:23.064+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:23.351 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68157]: 2024-04-26 22:14:23.13328604 +0000 UTC m=+5.885683853 container cleanup 377cc377708ad980c31e64a19a56dd4f0ac1f1d053a251054c16f92327a47eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:14:23.351 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 bash[68157]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4 2024-04-26T22:14:23.351 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:23 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:23.175+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:23.351 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:23 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:23.175+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:23.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:23 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:23.268+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:23.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:23 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:23.102+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:23.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:23 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:23.102+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:23.732 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68188]: 2024-04-26 22:14:23.351160717 +0000 UTC m=+0.570747474 container remove 377cc377708ad980c31e64a19a56dd4f0ac1f1d053a251054c16f92327a47eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:14:24.053 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:23 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:23.960+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:24.054 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68276]: 2024-04-26 22:14:23.650050509 +0000 UTC m=+0.018005949 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:24.054 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68276]: 2024-04-26 22:14:23.777085149 +0000 UTC m=+0.145040561 container create ccd95f5e49575b2c23cd728c2b3dc0e3a51da9e9fc1b6005a930dec600c62668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:24.054 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68276]: 2024-04-26 22:14:23.971017293 +0000 UTC m=+0.338972709 container init ccd95f5e49575b2c23cd728c2b3dc0e3a51da9e9fc1b6005a930dec600c62668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T22:14:24.054 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:23 smithi077 podman[68276]: 2024-04-26 22:14:23.974328268 +0000 UTC m=+0.342283691 container start ccd95f5e49575b2c23cd728c2b3dc0e3a51da9e9fc1b6005a930dec600c62668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T22:14:24.054 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:24 smithi077 podman[68276]: 2024-04-26 22:14:24.054412189 +0000 UTC m=+0.422367618 container attach ccd95f5e49575b2c23cd728c2b3dc0e3a51da9e9fc1b6005a930dec600c62668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T22:14:24.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:23 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:23.735+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:24.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:23 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:23.735+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:24.100 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:23 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:23.655+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:24.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:24 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:24.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:24.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:24 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:24.101+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:24.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:24 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:24.101+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:24.483 INFO:journalctl@ceph.osd.4.smithi077.stdout:Apr 26 22:14:24 smithi077 podman[68276]: 2024-04-26 22:14:24.265165113 +0000 UTC m=+0.633120537 container died ccd95f5e49575b2c23cd728c2b3dc0e3a51da9e9fc1b6005a930dec600c62668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:24.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:24 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:24.173+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:24.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:24 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:24.173+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:24.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:24.090+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:24.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:24.090+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:24.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T22:14:24.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:24.956 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T22:14:24.956 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T22:14:24.956 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.5 2024-04-26T22:14:24.983 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:24 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:24.674+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:25.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:24.732+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:25.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:24.732+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:25.148 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:24 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:24.930+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:25.264 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 systemd[1]: Stopping Ceph osd.5 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:25.264 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:25.131+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:25.264 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:25.131+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:25.265 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:25.264+0000 7f19ab487700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:25.265 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:25.264+0000 7f19ab487700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-26T22:14:25.265 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:25.264+0000 7f19ab487700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:25.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:25 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:25.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:25.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:25 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:25.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:25.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:25 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:25.131+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:25.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:25 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:25.131+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:25.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:25.314+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:26.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:25 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:25.773+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:26.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:25 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:25.773+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:26.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:25 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:25.773+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:26.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:25 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:25.702+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:26.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:25 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:25.915+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:26.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:26.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:26.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:26.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:26.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:26.139+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:26.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:26.139+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:26.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:26.139+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:26.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:26.298+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:26.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:26.172+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:26.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:26.172+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:27.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:26.784+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:27.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:26.784+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:27.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:26 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:26.784+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:27.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:26 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:26.657+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:27.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:26 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:26.951+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:27.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:27.138+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:27.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:27.138+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:27.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:27.138+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:27.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:27.163+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:27.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:27.163+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:27.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:27.163+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:27.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:27.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:27.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:27.197+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:27.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:27.197+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:27.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:27.197+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.677252+0000 front 2024-04-26T22:14:01.677272+0000 (oldest deadline 2024-04-26T22:14:26.377446+0000) 2024-04-26T22:14:28.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:27.765+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:28.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:27.765+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:28.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:27 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:27.765+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:28.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:27 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:27.644+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:28.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:27.951+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:28.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:27 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:27.951+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:28.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:28.144+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:28.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:28.144+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:28.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:28.144+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:28.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:28.197+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:28.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:28.197+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:28.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:28.197+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:28.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:28.308+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:28.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:28.222+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:28.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:28.222+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:28.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:28.222+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.677252+0000 front 2024-04-26T22:14:01.677272+0000 (oldest deadline 2024-04-26T22:14:26.377446+0000) 2024-04-26T22:14:29.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:28.743+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:29.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:28.743+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:29.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:28.743+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:29.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:28.688+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:29.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:28 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:28.688+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:29.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:28.951+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:29.233 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:28 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:28.951+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:29.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:29.146+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:29.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:29.146+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:29.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:29.146+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:29.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:29.233+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:29.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:29.233+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:29.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:29.233+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:29.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:29.352+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:29.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:29.268+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:29.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:29.268+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:29.641 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:29.268+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.677252+0000 front 2024-04-26T22:14:01.677272+0000 (oldest deadline 2024-04-26T22:14:26.377446+0000) 2024-04-26T22:14:30.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:29.771+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:30.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:29.771+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:30.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:29 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:29.771+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:30.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:29.665+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:30.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:29 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:29.665+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:30.482 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:29.989+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:30.482 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:29 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:29.989+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:30.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:30.119+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:30.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:30.119+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:30.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:30.119+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:30.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:30.239+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:30.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:30.239+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:30.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:30.239+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:30.639 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:30.384+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:30.640 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:30.247+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:43.873420+0000 front 2024-04-26T22:13:43.873281+0000 (oldest deadline 2024-04-26T22:14:08.573056+0000) 2024-04-26T22:14:30.640 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:30.247+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.576186+0000 front 2024-04-26T22:13:56.576064+0000 (oldest deadline 2024-04-26T22:14:17.676131+0000) 2024-04-26T22:14:30.640 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5[44700]: 2024-04-26T22:14:30.247+0000 7f19a3a73700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.677252+0000 front 2024-04-26T22:14:01.677272+0000 (oldest deadline 2024-04-26T22:14:26.377446+0000) 2024-04-26T22:14:30.890 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:30 smithi154 podman[69275]: 2024-04-26 22:14:30.735806519 +0000 UTC m=+5.586136775 container died 3173c977f3d2e3d4b77a010ba7911f5d9b4ed49d37cb1be5970b1416798a368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:14:30.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:30.640+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:30.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:30 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:30.640+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:31.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:30.744+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:31.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:30.744+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:31.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:30 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:30.744+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:31.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:30.963+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:31.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:30 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:30.963+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:31.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:31.075+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:31.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:31.075+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:31.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:31.075+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:31.390 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:31 smithi154 podman[69275]: 2024-04-26 22:14:31.141867026 +0000 UTC m=+5.992197318 container cleanup 3173c977f3d2e3d4b77a010ba7911f5d9b4ed49d37cb1be5970b1416798a368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T22:14:31.391 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:31 smithi154 bash[69275]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5 2024-04-26T22:14:31.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:31.209+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:31.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:31.209+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:31.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:31.209+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:31.684 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:31 smithi154 podman[69293]: 2024-04-26 22:14:31.484689575 +0000 UTC m=+0.744256087 container remove 3173c977f3d2e3d4b77a010ba7911f5d9b4ed49d37cb1be5970b1416798a368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:14:31.684 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:31.430+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:31.684 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:31.430+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:31.936 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:31 smithi154 podman[69384]: 2024-04-26 22:14:31.680342226 +0000 UTC m=+0.017936292 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:31.936 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:31 smithi154 podman[69384]: 2024-04-26 22:14:31.827241503 +0000 UTC m=+0.164835554 container create d09a67d8267578452db6a0d11c0a4c1fc28c7a48f471a8f1c70b7fe81c458c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:31.936 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:31.683+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:31.936 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:31 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:31.683+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:32.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:31.774+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:32.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:31.774+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:32.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:31 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:31.774+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:32.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:31.949+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:32.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:31 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:31.949+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:32.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:32.068+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:32.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:32.068+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:32.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:32.068+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:32.363 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:32 smithi154 podman[69384]: 2024-04-26 22:14:32.063588395 +0000 UTC m=+0.401182460 container init d09a67d8267578452db6a0d11c0a4c1fc28c7a48f471a8f1c70b7fe81c458c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:32.363 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:32 smithi154 podman[69384]: 2024-04-26 22:14:32.067287465 +0000 UTC m=+0.404881520 container start d09a67d8267578452db6a0d11c0a4c1fc28c7a48f471a8f1c70b7fe81c458c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:14:32.363 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:32 smithi154 podman[69384]: 2024-04-26 22:14:32.163321049 +0000 UTC m=+0.500915109 container attach d09a67d8267578452db6a0d11c0a4c1fc28c7a48f471a8f1c70b7fe81c458c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T22:14:32.363 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 26 22:14:32 smithi154 podman[69384]: 2024-04-26 22:14:32.364064241 +0000 UTC m=+0.701658308 container died d09a67d8267578452db6a0d11c0a4c1fc28c7a48f471a8f1c70b7fe81c458c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T22:14:32.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:32.168+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:32.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:32.168+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:32.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:32.168+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:32.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:32.439+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:32.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:32.439+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:33.041 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:32.731+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:33.041 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:32 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:32.731+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:33.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T22:14:33.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:32.763+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:33.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:32.763+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:33.073 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:32 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:32.763+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:33.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:33.144 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T22:14:33.144 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T22:14:33.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.6 2024-04-26T22:14:33.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:32.974+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:33.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:32 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:32.974+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:33.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:33 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:33.088+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:33.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:33 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:33.088+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:33.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:33 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:33.088+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:33.573 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 systemd[1]: Stopping Ceph osd.6 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:33.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:33.175+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:33.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:33.175+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:33.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:33.175+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:33.737 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:33 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:33.469+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:33.737 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:33 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:33.469+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.718+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.718+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.718+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.785+0000 7efc05200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.785+0000 7efc05200700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-26T22:14:34.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:33.785+0000 7efc05200700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:34.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:33 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:33.736+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:34.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:33 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:33.736+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:34.482 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:34.005+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:34.482 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:34.005+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:34.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:34.122+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:34.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:34.122+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:34.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:34.122+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:34.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:34.129+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:34.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:34.129+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:34.574 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:34.129+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:34.711+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:34.711+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:34.711+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:34.484+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:34.484+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:34.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:34 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:34.484+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:35.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:34.706+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:35.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:34.706+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:35.075 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:34 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:34.706+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:35.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:34.964+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:35.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:34 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:34.964+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:35.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:35.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:35.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:35.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:35.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:35.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:35.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:35.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:35.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:35.094+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:35.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:35.094+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:35.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:35.094+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:35.709 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:35.434+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:35.709 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:35.434+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:35.709 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:35.434+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:36.050 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:35.688+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:36.050 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:35.688+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:36.051 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:35 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:35.688+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:36.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:35.708+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:36.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:35.708+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:36.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:35 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:35.708+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:36.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:35.960+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:36.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:35.960+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:36.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:35 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:35.960+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:36.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:36.167+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:36.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:36.167+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:36.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:36.167+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:36.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:36.167+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:36.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:36.049+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:36.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:36.049+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:36.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:36.049+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:36.715 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:36.463+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:36.715 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:36.463+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:36.715 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:36.463+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:37.029 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:36.719+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:37.030 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:36.719+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:37.030 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:36.719+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:37.030 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:36.719+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.884488+0000 front 2024-04-26T22:14:07.884779+0000 (oldest deadline 2024-04-26T22:14:36.684778+0000) 2024-04-26T22:14:37.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:36.715+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:37.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:36.715+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:37.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:36 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:36.715+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:37.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:36.919+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:37.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:36.919+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:37.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:36 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:36.919+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:37.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:37.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:37.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:37.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:37.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:37.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:37.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:37.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:37.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.028+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:37.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.028+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:37.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.028+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:37.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.028+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:37.703 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:37.430+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:37.703 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:37.430+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:37.703 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:37.430+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:38.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.988+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.988+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.988+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:37.988+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:37.740+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.777200+0000 front 2024-04-26T22:13:45.777328+0000 (oldest deadline 2024-04-26T22:14:11.077312+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:37.740+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.080202+0000 front 2024-04-26T22:13:55.080394+0000 (oldest deadline 2024-04-26T22:14:17.980127+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:37.740+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.481861+0000 front 2024-04-26T22:14:01.482113+0000 (oldest deadline 2024-04-26T22:14:25.581861+0000) 2024-04-26T22:14:38.074 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:37 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6[56473]: 2024-04-26T22:14:37.740+0000 7efc00a00700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.884488+0000 front 2024-04-26T22:14:07.884779+0000 (oldest deadline 2024-04-26T22:14:36.684778+0000) 2024-04-26T22:14:38.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:37.703+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:38.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:37.703+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:38.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:37 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:37.703+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:38.142 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:37.885+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:38.142 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:37.885+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:38.142 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:37 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:37.885+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:38.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:38.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:38.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:38.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:38.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:38.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:38.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:38.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:38.668 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:38.412+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:38.668 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:38.412+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:38.669 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:38.412+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:39.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:38.668+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:39.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:38.668+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:39.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:38 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:38.668+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:39.156 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:38.839+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:39.156 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:38.839+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:39.156 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:38 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:38.839+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:39.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.018+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:39.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.018+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:39.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.018+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:39.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.018+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:39.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:39.156+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:39.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:39.156+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:39.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:39.156+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:39.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:39.156+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:39.626 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:39 smithi012 podman[89017]: 2024-04-26 22:14:39.349365223 +0000 UTC m=+5.724055101 container died 1e36a681767bdb80866d0ec2b34869b7035e22c9c8be5ed5e96f7ea04ff8460a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T22:14:39.638 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:39.377+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:39.638 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:39.377+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:39.638 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:39.377+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:39.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:39.638+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:39.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:39.638+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:39.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:39 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:39.638+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:39.971 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:39 smithi012 podman[89017]: 2024-04-26 22:14:39.744631086 +0000 UTC m=+6.119320970 container cleanup 1e36a681767bdb80866d0ec2b34869b7035e22c9c8be5ed5e96f7ea04ff8460a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T22:14:39.971 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:39 smithi012 bash[89017]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6 2024-04-26T22:14:40.113 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:39.852+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:40.113 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:39.852+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:40.113 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:39 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:39.852+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:40.323 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:39 smithi012 podman[89035]: 2024-04-26 22:14:39.970602768 +0000 UTC m=+0.607886582 container remove 1e36a681767bdb80866d0ec2b34869b7035e22c9c8be5ed5e96f7ea04ff8460a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T22:14:40.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.971+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:40.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.971+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:40.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.971+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:40.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:39 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:39.971+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:40.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:40.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:40.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:40.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:40.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:40.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:40.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:40.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:40.633 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:40.366+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:40.633 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:40.366+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:40.633 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:40.366+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:40.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:40.632+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:40.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:40.632+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:40.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:40 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:40.632+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:41.117 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:40.865+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:41.117 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:40.865+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:41.117 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:40.865+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:41.193 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:40.985+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:41.193 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:40.985+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:41.193 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:40.985+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:41.193 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:40 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:40.985+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:41.193 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:40 smithi012 podman[89125]: 2024-04-26 22:14:40.730715133 +0000 UTC m=+0.033121734 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:41.194 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:40 smithi012 podman[89125]: 2024-04-26 22:14:40.888210447 +0000 UTC m=+0.190617088 container create 210147b77ae5b35d1b988dbbd06a451ad7f1110b3745c35372fcfca554948100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T22:14:41.194 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:41 smithi012 podman[89125]: 2024-04-26 22:14:41.192200996 +0000 UTC m=+0.494607641 container init 210147b77ae5b35d1b988dbbd06a451ad7f1110b3745c35372fcfca554948100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T22:14:41.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:41.117+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:41.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:41.117+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:41.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:41.117+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:41.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:41.117+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:41.573 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:41 smithi012 podman[89125]: 2024-04-26 22:14:41.204108307 +0000 UTC m=+0.506514915 container start 210147b77ae5b35d1b988dbbd06a451ad7f1110b3745c35372fcfca554948100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T22:14:41.573 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:41 smithi012 podman[89125]: 2024-04-26 22:14:41.307470329 +0000 UTC m=+0.609877000 container attach 210147b77ae5b35d1b988dbbd06a451ad7f1110b3745c35372fcfca554948100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T22:14:41.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:41.358+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:41.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:41.358+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:41.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:41.358+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:42.018 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 26 22:14:41 smithi012 podman[89125]: 2024-04-26 22:14:41.672039812 +0000 UTC m=+0.974446425 container died 210147b77ae5b35d1b988dbbd06a451ad7f1110b3745c35372fcfca554948100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T22:14:42.125 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:41.821+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:42.125 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:41.821+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:42.125 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:41 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:41.821+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:42.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:41.671+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:42.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:41.671+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:42.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:41 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:41.671+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:42.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:42.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:42.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:42.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:42.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:42.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:42.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:42 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:42.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:42.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:42.124+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:42.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:42.124+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:42.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:42.124+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:42.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:42.124+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:42.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T22:14:42.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:42.626 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T22:14:42.627 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T22:14:42.627 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.7 2024-04-26T22:14:42.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:42.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:42.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:42.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:42.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:42.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 systemd[1]: Stopping Ceph osd.7 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.789+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.789+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.789+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.943+0000 7fa9158a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.943+0000 7fa9158a2700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-26T22:14:42.944 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:42 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:42.943+0000 7fa9158a2700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:43.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:42.699+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:43.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:42.699+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:43.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:42 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:42.699+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:43.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:43.129+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:43.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:43.129+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:43.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:43.129+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:43.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:43.129+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:43.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:43 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:43.001+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:43.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:43 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:43.001+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:43.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:43 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:43.001+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:43.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:43 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:43.001+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:43.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:43.346+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:43.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:43.346+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:43.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:43.346+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:43.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:43.346+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:43.734 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:43.733+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:44.118 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:43.775+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:44.119 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:43.775+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:44.119 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:43 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:43.775+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:44.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:43.733+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:44.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:43 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:43.733+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:44.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:44.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:44.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:44.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:44.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.016+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:44.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:44.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:44.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:44.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:44.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:44.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:44.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:44.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:44.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:44.118+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:44.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:44.343+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:44.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:44.343+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:44.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:44.343+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:44.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:44.343+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:45.078 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:44.810+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:45.078 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:44.810+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:45.079 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:44.810+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:45.079 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:44.810+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:18.006694+0000 front 2024-04-26T22:14:18.006738+0000 (oldest deadline 2024-04-26T22:14:43.906888+0000) 2024-04-26T22:14:45.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:44.783+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:45.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:44.783+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:45.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:44 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:44.783+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:45.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:45.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:45.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:45.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:45.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:44 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:44.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:45.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:45.078+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:45.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:45.078+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:45.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:45.078+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:45.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:45.078+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:45.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:45.078+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:45.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:45.387+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:45.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:45.387+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:45.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:45.387+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:45.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:45.387+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:46.128 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:45.831+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:46.128 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:45.831+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:46.128 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:45.831+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:46.128 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:45 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:45.831+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:18.006694+0000 front 2024-04-26T22:14:18.006738+0000 (oldest deadline 2024-04-26T22:14:43.906888+0000) 2024-04-26T22:14:46.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:45.763+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:46.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:45.763+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:46.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:45.763+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:46.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:45 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:45.763+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:46.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:45.991+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:46.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:45.991+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:46.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:45.991+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:46.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:45.991+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:46.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:45 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:45.991+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:46.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:46.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:46.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:46.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:46.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:46.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:46.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:46.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:46.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:46.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:46.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:46.390+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:46.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:46.390+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:46.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:46.390+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:46.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:46.390+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:47.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:46.726+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:47.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:46.726+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:47.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:46.726+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:47.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:46 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:46.726+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:47.176 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:46.828+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:47.176 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:46.828+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:47.176 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:46.828+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:47.176 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:46 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:46.828+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:18.006694+0000 front 2024-04-26T22:14:18.006738+0000 (oldest deadline 2024-04-26T22:14:43.906888+0000) 2024-04-26T22:14:47.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:46.974+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:47.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:46.974+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:47.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:46.974+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:47.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:46.974+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:47.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:46 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:46.974+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:47.442 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:47.176+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:47.442 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:47.176+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:47.442 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:47.176+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:47.442 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:47.176+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:47.442 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:47.176+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:47.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:47.372+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:47.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:47.372+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:47.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:47.372+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:47.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:47.372+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:48.130 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:47.840+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.203344+0000 front 2024-04-26T22:13:55.203373+0000 (oldest deadline 2024-04-26T22:14:18.703472+0000) 2024-04-26T22:14:48.130 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:47.840+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.004677+0000 front 2024-04-26T22:14:04.004807+0000 (oldest deadline 2024-04-26T22:14:27.504527+0000) 2024-04-26T22:14:48.130 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:47.840+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.005955+0000 front 2024-04-26T22:14:11.005817+0000 (oldest deadline 2024-04-26T22:14:35.705639+0000) 2024-04-26T22:14:48.131 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7[50365]: 2024-04-26T22:14:47.840+0000 7fa90de8e700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:18.006694+0000 front 2024-04-26T22:14:18.006738+0000 (oldest deadline 2024-04-26T22:14:43.906888+0000) 2024-04-26T22:14:48.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:47.740+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:48.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:47.740+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:48.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:47.740+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:48.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:47 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:47.740+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:48.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:47.963+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:48.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:47.963+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:48.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:47.963+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:48.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:47.963+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:48.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:47 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:47.963+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:48 smithi077 podman[68381]: 2024-04-26 22:14:48.30716833 +0000 UTC m=+5.484943506 container died 73568df389dfb99c28247315cee4647a7cd7a07bbbb51b90a4429711ee512b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:48 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:48.130+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:48 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:48.130+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:48 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:48.130+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:48 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:48.130+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:48.385 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:48 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:48.130+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:48.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:48.334+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:48.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:48.334+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:48.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:48.334+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:48.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:48.334+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:48.969 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:48 smithi077 podman[68381]: 2024-04-26 22:14:48.73601893 +0000 UTC m=+5.913794154 container cleanup 73568df389dfb99c28247315cee4647a7cd7a07bbbb51b90a4429711ee512b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T22:14:48.970 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:48 smithi077 bash[68381]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7 2024-04-26T22:14:49.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:48.700+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:49.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:48.700+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:49.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:48.700+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:49.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:48 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:48.700+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:49.232 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:48 smithi077 podman[68409]: 2024-04-26 22:14:48.970250966 +0000 UTC m=+0.665498695 container remove 73568df389dfb99c28247315cee4647a7cd7a07bbbb51b90a4429711ee512b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:49.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:49 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:49.160+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:49.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:49 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:49.160+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:49.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:49 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:49.160+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:49.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:49 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:49.160+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:49.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:49 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:49.160+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:49.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:48.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:49.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:48.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:49.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:48.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:49.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:48.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:49.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:48 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:48.986+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:49.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:49.360+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:49.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:49.360+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:49.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:49.360+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:49.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:49.360+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:49.664 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.292993863 +0000 UTC m=+0.017832302 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:49.664 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.429455397 +0000 UTC m=+0.154293832 container create 54afe8f60cf61d43975dd77928958b8de773e6e450799df8b07d533fff1d86b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T22:14:49.664 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.665021562 +0000 UTC m=+0.389859989 container init 54afe8f60cf61d43975dd77928958b8de773e6e450799df8b07d533fff1d86b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T22:14:49.953 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.668195036 +0000 UTC m=+0.393033469 container start 54afe8f60cf61d43975dd77928958b8de773e6e450799df8b07d533fff1d86b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:14:49.953 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.765095992 +0000 UTC m=+0.489934445 container attach 54afe8f60cf61d43975dd77928958b8de773e6e450799df8b07d533fff1d86b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:14:49.953 INFO:journalctl@ceph.osd.7.smithi077.stdout:Apr 26 22:14:49 smithi077 podman[68498]: 2024-04-26 22:14:49.953732371 +0000 UTC m=+0.678570796 container died 54afe8f60cf61d43975dd77928958b8de773e6e450799df8b07d533fff1d86b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T22:14:50.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:49.651+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:50.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:49.651+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:50.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:49.651+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:50.141 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:49 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:49.651+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:50.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:50 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:50.116+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:50.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:50 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:50.116+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:50.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:50 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:50.116+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:50.232 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:50 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:50.116+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:50.233 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:50 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:50.116+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:50.258 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:49.954+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:50.258 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:49.954+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:50.258 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:49.954+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:50.258 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:49.954+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:50.258 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:49 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:49.954+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:50.569 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T22:14:50.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:50.611 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T22:14:50.611 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T22:14:50.611 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.8 2024-04-26T22:14:50.637 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:50.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:50.637 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:50.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:50.637 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:50.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:50.637 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:50.379+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:50.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.652+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:50.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.652+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:50.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.652+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:50.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.652+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:50.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 systemd[1]: Stopping Ceph osd.8 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:51.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:50.934+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:51.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:50.934+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:51.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:50.934+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:51.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:50.934+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:51.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:50 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:50.934+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:51.388 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.935+0000 7f161a532700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:51.388 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.935+0000 7f161a532700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-26T22:14:51.388 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:50 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:50.935+0000 7f161a532700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:51.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:51.105+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:51.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:51.105+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:51.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:51.105+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:51.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:51.105+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:51.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:51 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:51.105+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:51.640 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:51.622+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:51.622+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:51.622+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:51.622+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:51.388+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:51.388+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:51.388+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:51.388+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:51.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:51 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:51.388+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:52.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:52.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:52.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:52.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:52.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:52.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:51 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:51.908+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:52.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:52.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:52.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:52.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:52.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:52.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:52.087+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:52.399+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:52.399+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:52.399+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:52.399+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:52.399+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:52.608+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:52.608+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:52.891 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:52.608+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:52.892 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:52.608+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:52.892 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:52 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:52.608+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:28.845888+0000 front 2024-04-26T22:14:28.845928+0000 (oldest deadline 2024-04-26T22:14:51.646511+0000) 2024-04-26T22:14:53.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:53.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:53.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:53.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:53.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:53.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:52 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:52.880+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:53.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:53.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:53.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:53.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:53.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:53.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:53 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:53.040+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:53.598+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:53.598+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:53.598+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:53.598+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:53.598+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:28.845888+0000 front 2024-04-26T22:14:28.845928+0000 (oldest deadline 2024-04-26T22:14:51.646511+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:53.357+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:53.357+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:53.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:53.357+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:53.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:53.357+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:53.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:53 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:53.357+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:54.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:54.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:54.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:54.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:54.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:54.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:53 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:53.852+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:54.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:54.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:54.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:54.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:54.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:54.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:54 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:54.010+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:54.640 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:54.583+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:54.583+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:54.583+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:54.583+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:54.583+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:28.845888+0000 front 2024-04-26T22:14:28.845928+0000 (oldest deadline 2024-04-26T22:14:51.646511+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:54.381+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:54.381+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:54.381+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:54.381+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:54.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:54 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:54.381+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:55.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:55.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:55.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:55.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:55.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:55.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:54 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:54.864+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:55.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:55.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:55.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:55.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:55.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:55.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:55 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:55.032+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:55.640 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:55.537+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:56.240911+0000 front 2024-04-26T22:13:56.241040+0000 (oldest deadline 2024-04-26T22:14:18.540982+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:55.537+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.441917+0000 front 2024-04-26T22:14:04.442120+0000 (oldest deadline 2024-04-26T22:14:27.942077+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:55.537+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:07.942706+0000 front 2024-04-26T22:14:07.942941+0000 (oldest deadline 2024-04-26T22:14:33.842571+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:55.537+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:21.344579+0000 front 2024-04-26T22:14:21.344731+0000 (oldest deadline 2024-04-26T22:14:44.844460+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8[51545]: 2024-04-26T22:14:55.537+0000 7f1616b26700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:28.845888+0000 front 2024-04-26T22:14:28.845928+0000 (oldest deadline 2024-04-26T22:14:51.646511+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:55.356+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:55.356+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:55.356+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:55.356+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:55.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:55 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:55.356+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:56.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:56.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:56.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:56.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:56.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:56.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:55 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:55.862+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:56.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:56.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:56.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:56.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:56.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:56.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.012+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:56 smithi154 podman[69490]: 2024-04-26 22:14:56.306789669 +0000 UTC m=+5.500606447 container died cffb636643913e455bd65a79ce70d2b3e0e883cc6616c01853d8c9bed817a2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:56.329+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:56.329+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:56.329+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:56.329+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:56.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:56.329+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:56.828 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:56 smithi154 podman[69490]: 2024-04-26 22:14:56.644309127 +0000 UTC m=+5.838125913 container cleanup cffb636643913e455bd65a79ce70d2b3e0e883cc6616c01853d8c9bed817a2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:56.828 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:56 smithi154 bash[69490]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8 2024-04-26T22:14:57.140 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:56 smithi154 podman[69507]: 2024-04-26 22:14:56.828730693 +0000 UTC m=+0.523983884 container remove cffb636643913e455bd65a79ce70d2b3e0e883cc6616c01853d8c9bed817a2b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T22:14:57.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:57.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:57.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:57.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:57.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:57.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:56 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:56.838+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:57.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:57.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:57.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:57.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:57.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:57.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:56 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:56.995+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:57.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:57 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:57.354+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:57.572 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:57 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:57.354+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:57 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:57.354+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:57 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:57.354+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:57 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:57.354+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.157007268 +0000 UTC m=+0.017749777 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.262728548 +0000 UTC m=+0.123471024 container create b1fdbe23b5f6c3f0573edcdbf3162886c047a7ca63d5d07a41a5b6125ed98e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.481689223 +0000 UTC m=+0.342431700 container init b1fdbe23b5f6c3f0573edcdbf3162886c047a7ca63d5d07a41a5b6125ed98e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.485536508 +0000 UTC m=+0.346278988 container start b1fdbe23b5f6c3f0573edcdbf3162886c047a7ca63d5d07a41a5b6125ed98e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T22:14:57.573 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.573337703 +0000 UTC m=+0.434080189 container attach b1fdbe23b5f6c3f0573edcdbf3162886c047a7ca63d5d07a41a5b6125ed98e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:14:57.890 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 26 22:14:57 smithi154 podman[69596]: 2024-04-26 22:14:57.779554421 +0000 UTC m=+0.640296903 container died b1fdbe23b5f6c3f0573edcdbf3162886c047a7ca63d5d07a41a5b6125ed98e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:14:58.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:58.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:58.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:58.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:58.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:58.324 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:57 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:57.840+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:58.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:58 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:58.330+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:58.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:58 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:58.330+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:58.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:58 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:58.330+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:58.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:58 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:58.330+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:58.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:58 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:58.330+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:14:58.412 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T22:14:58.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:58.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:58.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:58.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:58.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:58.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:58 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:58.033+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:58.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:14:58.491 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T22:14:58.491 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T22:14:58.491 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.9 2024-04-26T22:14:59.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 systemd[1]: Stopping Ceph osd.9 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:14:59.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:14:59.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:14:59.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:14:59.075 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:14:59.075 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:14:59.075 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:58 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:58.801+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:14:59.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:14:59.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:14:59.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:14:59.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:14:59.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:14:59.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:14:59 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:14:59.046+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:14:59.574 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.118+0000 7feff5800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:14:59.574 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.118+0000 7feff5800700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-26T22:14:59.574 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.118+0000 7feff5800700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:14:59.603 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:59.335+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:14:59.603 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:59.335+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:14:59.603 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:59.335+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:14:59.603 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:59.335+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:14:59.603 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:14:59 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:14:59.335+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:15:00.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:14:59 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:14:59.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:15:00.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:00.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:00.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:00.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:00.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:00.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:00 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:00.039+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:00.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:00 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:00.288+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:01.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:15:01.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:15:01.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:15:01.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:15:01.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:15:01.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:00 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:00.752+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:15:01.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:01.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:01 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:01.049+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:01.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:01.245+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:02.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:15:02.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:15:02.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:15:02.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:15:02.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:15:02.075 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:01 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:01.711+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:15:02.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:02.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:02 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:02.084+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:02.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:02 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:02.283+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:03.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:15:03.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:15:03.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:15:03.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:15:03.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:15:03.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:02 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:02.748+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:15:03.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:03.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:03.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:03.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:03.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:03.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:03.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:03 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:03.113+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:03.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:03 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:03.277+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:04.073 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:45.890236+0000 front 2024-04-26T22:13:45.890356+0000 (oldest deadline 2024-04-26T22:14:08.190462+0000) 2024-04-26T22:15:04.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:55.193525+0000 front 2024-04-26T22:13:55.193478+0000 (oldest deadline 2024-04-26T22:14:20.493644+0000) 2024-04-26T22:15:04.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:00.494724+0000 front 2024-04-26T22:14:00.494525+0000 (oldest deadline 2024-04-26T22:14:25.794866+0000) 2024-04-26T22:15:04.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:11.598193+0000 front 2024-04-26T22:14:11.598245+0000 (oldest deadline 2024-04-26T22:14:36.298120+0000) 2024-04-26T22:15:04.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.400952+0000 front 2024-04-26T22:14:17.401165+0000 (oldest deadline 2024-04-26T22:14:43.300282+0000) 2024-04-26T22:15:04.074 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:03 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9[63185]: 2024-04-26T22:15:03.703+0000 7feff1000700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:26.102895+0000 front 2024-04-26T22:14:26.102953+0000 (oldest deadline 2024-04-26T22:14:51.402628+0000) 2024-04-26T22:15:04.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:04.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:04.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:04.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:04.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:04.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:04.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:04 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:04.164+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:04.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:04 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:04.230+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:05.043 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:04 smithi012 podman[89243]: 2024-04-26 22:15:04.757262156 +0000 UTC m=+5.806203168 container died df0a3ab05fcd6962b113f65b02c4d641f739f4e65c38a82e0a778d06c35fca62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T22:15:05.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:05 smithi012 podman[89243]: 2024-04-26 22:15:05.161484067 +0000 UTC m=+6.210425117 container cleanup df0a3ab05fcd6962b113f65b02c4d641f739f4e65c38a82e0a778d06c35fca62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:15:05.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:05 smithi012 bash[89243]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9 2024-04-26T22:15:05.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:05.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:05.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:05.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:05.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:05.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:05.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:05 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:05.168+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:05.640 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:05.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:05.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:05.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:05.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:05.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:05 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:05.211+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:05.823 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:05 smithi012 podman[89262]: 2024-04-26 22:15:05.421004268 +0000 UTC m=+0.650625287 container remove df0a3ab05fcd6962b113f65b02c4d641f739f4e65c38a82e0a778d06c35fca62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T22:15:06.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:06.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:06.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:06.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:06.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:06.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:06.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:06 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:06.142+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:06.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:06 smithi012 podman[89349]: 2024-04-26 22:15:06.165127291 +0000 UTC m=+0.038110962 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:15:06.573 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:06 smithi012 podman[89349]: 2024-04-26 22:15:06.272178356 +0000 UTC m=+0.145162064 container create 83f2f03acb20dca9fa3d298d4294872255c9e15c4f06a86739685c5073627e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T22:15:06.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:06.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:06.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:06.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:06.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:06.642 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:06.179+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:07.046 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:06 smithi012 podman[89349]: 2024-04-26 22:15:06.585412126 +0000 UTC m=+0.458395811 container init 83f2f03acb20dca9fa3d298d4294872255c9e15c4f06a86739685c5073627e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T22:15:07.046 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:06 smithi012 podman[89349]: 2024-04-26 22:15:06.596030737 +0000 UTC m=+0.469014788 container start 83f2f03acb20dca9fa3d298d4294872255c9e15c4f06a86739685c5073627e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:15:07.046 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:06 smithi012 podman[89349]: 2024-04-26 22:15:06.674463159 +0000 UTC m=+0.547446832 container attach 83f2f03acb20dca9fa3d298d4294872255c9e15c4f06a86739685c5073627e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:15:07.323 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 26 22:15:07 smithi012 podman[89349]: 2024-04-26 22:15:07.092747682 +0000 UTC m=+0.965731396 container died 83f2f03acb20dca9fa3d298d4294872255c9e15c4f06a86739685c5073627e4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True) 2024-04-26T22:15:07.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:07.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:07 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:07.128+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:07.641 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:07 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:07.153+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:07.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T22:15:08.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:08.035 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T22:15:08.035 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T22:15:08.035 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.10 2024-04-26T22:15:08.363 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 systemd[1]: Stopping Ceph osd.10 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:08.363 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:08.363 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:08.363 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.127+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.362+0000 7fd80be6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.362+0000 7fd80be6b700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-26T22:15:08.364 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:08 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:08.362+0000 7fd80be6b700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:08.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:08 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:08.112+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:09.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:09 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:09.119+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:09.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:09.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:09 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:09.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:10.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:10 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:10.092+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:10.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:10.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:10.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:10.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:10.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:10.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:10.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:10 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:10.115+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:11.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:11.057+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:11.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:11.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:11.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:11.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:11.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:11.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:11.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:11 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:11.114+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:12.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:12.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:12 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:12.090+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:12.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:12.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:12.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:12.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:12.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:12.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:12.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:12 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:12.152+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:13.394 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:13 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:13.070+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:13.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:44.831457+0000 front 2024-04-26T22:13:44.831661+0000 (oldest deadline 2024-04-26T22:14:10.131192+0000) 2024-04-26T22:15:13.482 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-04-26T22:13:53.632215+0000 front 2024-04-26T22:13:53.632345+0000 (oldest deadline 2024-04-26T22:14:18.932337+0000) 2024-04-26T22:15:13.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:04.335903+0000 front 2024-04-26T22:14:04.335824+0000 (oldest deadline 2024-04-26T22:14:27.136259+0000) 2024-04-26T22:15:13.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.238850+0000 front 2024-04-26T22:14:12.238870+0000 (oldest deadline 2024-04-26T22:14:34.538472+0000) 2024-04-26T22:15:13.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:19.139843+0000 front 2024-04-26T22:14:19.139818+0000 (oldest deadline 2024-04-26T22:14:43.239893+0000) 2024-04-26T22:15:13.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:29.641344+0000 front 2024-04-26T22:14:29.641403+0000 (oldest deadline 2024-04-26T22:14:51.941323+0000) 2024-04-26T22:15:13.483 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10[57020]: 2024-04-26T22:15:13.148+0000 7fd804457700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:37.242649+0000 front 2024-04-26T22:14:37.242748+0000 (oldest deadline 2024-04-26T22:15:00.742442+0000) 2024-04-26T22:15:14.114 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:13 smithi077 podman[68603]: 2024-04-26 22:15:13.832765795 +0000 UTC m=+5.602006138 container died b087d687d93387b9372ce57301c9c4d83e1225127310a3dbfdada68f4938891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:14.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:14 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:14.074+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:14.472 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:14 smithi077 podman[68603]: 2024-04-26 22:15:14.20498789 +0000 UTC m=+5.974228233 container cleanup b087d687d93387b9372ce57301c9c4d83e1225127310a3dbfdada68f4938891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T22:15:14.472 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:14 smithi077 bash[68603]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10 2024-04-26T22:15:14.732 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:14 smithi077 podman[68620]: 2024-04-26 22:15:14.473245443 +0000 UTC m=+0.638687016 container remove b087d687d93387b9372ce57301c9c4d83e1225127310a3dbfdada68f4938891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T22:15:15.084 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:14 smithi077 podman[68710]: 2024-04-26 22:15:14.719798794 +0000 UTC m=+0.018119820 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:15:15.085 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:14 smithi077 podman[68710]: 2024-04-26 22:15:14.85781116 +0000 UTC m=+0.156132188 container create 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:15:15.085 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 podman[68710]: 2024-04-26 22:15:15.085149334 +0000 UTC m=+0.383470361 container init 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T22:15:15.371 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 podman[68710]: 2024-04-26 22:15:15.08823908 +0000 UTC m=+0.386560109 container start 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T22:15:15.371 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 podman[68710]: 2024-04-26 22:15:15.160030483 +0000 UTC m=+0.458351527 container attach 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:15:15.390 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:15.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:15 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:15.067+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:15.645 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 podman[68710]: 2024-04-26 22:15:15.371797892 +0000 UTC m=+0.670118921 container died 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T22:15:15.962 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 podman[68736]: 2024-04-26 22:15:15.937804374 +0000 UTC m=+0.560713140 container remove 6b75896a905d18217d545a1c95bc8087e68a0a4e3b578d9c80cd2cf011b94ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T22:15:15.963 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-26T22:15:15.963 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 systemd[1]: Stopped Ceph osd.10 for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T22:15:15.963 INFO:journalctl@ceph.osd.10.smithi077.stdout:Apr 26 22:15:15 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.10.service: Consumed 31.598s CPU time. 2024-04-26T22:15:15.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T22:15:16.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:16.003 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T22:15:16.004 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T22:15:16.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.11 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.039+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 systemd[1]: Stopping Ceph osd.11 for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:16.288 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.287+0000 7f155d7b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:16.289 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.287+0000 7f155d7b4700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-26T22:15:16.289 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:16 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:16.287+0000 7f155d7b4700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:15:17.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:17.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:17.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:17.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:17.392 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:17.392 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:17.392 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:17.392 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.002+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-04-26T22:14:50.761727+0000 front 2024-04-26T22:14:50.761744+0000 (oldest deadline 2024-04-26T22:15:16.661723+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:18.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:17 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:17.994+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-04-26T22:14:50.761727+0000 front 2024-04-26T22:14:50.761744+0000 (oldest deadline 2024-04-26T22:15:16.661723+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:19.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:18 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:18.967+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-04-26T22:14:50.761727+0000 front 2024-04-26T22:14:50.761744+0000 (oldest deadline 2024-04-26T22:15:16.661723+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:20.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:19 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:19.970+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-04-26T22:14:50.761727+0000 front 2024-04-26T22:14:50.761744+0000 (oldest deadline 2024-04-26T22:15:16.661723+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-26T22:13:46.350569+0000 front 2024-04-26T22:13:46.350732+0000 (oldest deadline 2024-04-26T22:14:08.050422+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-26T22:14:01.953779+0000 front 2024-04-26T22:14:01.953644+0000 (oldest deadline 2024-04-26T22:14:30.754342+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-26T22:14:12.455711+0000 front 2024-04-26T22:14:12.455645+0000 (oldest deadline 2024-04-26T22:14:34.155562+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-04-26T22:14:17.656622+0000 front 2024-04-26T22:14:17.656684+0000 (oldest deadline 2024-04-26T22:14:42.956666+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-26T22:14:25.857270+0000 front 2024-04-26T22:14:25.857299+0000 (oldest deadline 2024-04-26T22:14:51.157391+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-26T22:14:35.659613+0000 front 2024-04-26T22:14:35.659538+0000 (oldest deadline 2024-04-26T22:14:59.759371+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-04-26T22:14:46.061554+0000 front 2024-04-26T22:14:46.061614+0000 (oldest deadline 2024-04-26T22:15:10.761376+0000) 2024-04-26T22:15:21.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:20 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11[58145]: 2024-04-26T22:15:20.957+0000 7f1555da0700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-04-26T22:14:50.761727+0000 front 2024-04-26T22:14:50.761744+0000 (oldest deadline 2024-04-26T22:15:16.661723+0000) 2024-04-26T22:15:22.003 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:21 smithi154 podman[69714]: 2024-04-26 22:15:21.683771901 +0000 UTC m=+5.490174544 container died 4242eeda48792a25409c1088b797948e9e665cfd9e760abe62370a140d31028f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T22:15:22.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69714]: 2024-04-26 22:15:22.004279316 +0000 UTC m=+5.810681957 container cleanup 4242eeda48792a25409c1088b797948e9e665cfd9e760abe62370a140d31028f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:15:22.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 bash[69714]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11 2024-04-26T22:15:22.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69731]: 2024-04-26 22:15:22.17225328 +0000 UTC m=+0.487613036 container remove 4242eeda48792a25409c1088b797948e9e665cfd9e760abe62370a140d31028f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:15:22.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69820]: 2024-04-26 22:15:22.513586728 +0000 UTC m=+0.018439188 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:15:22.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69820]: 2024-04-26 22:15:22.631521006 +0000 UTC m=+0.136373448 container create a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:15:22.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69820]: 2024-04-26 22:15:22.850629698 +0000 UTC m=+0.355482170 container init a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T22:15:22.891 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69820]: 2024-04-26 22:15:22.853900048 +0000 UTC m=+0.358752492 container start a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T22:15:23.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:22 smithi154 podman[69820]: 2024-04-26 22:15:22.933679202 +0000 UTC m=+0.438531660 container attach a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:15:23.391 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:23 smithi154 podman[69820]: 2024-04-26 22:15:23.142666118 +0000 UTC m=+0.647518553 container died a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T22:15:23.683 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 26 22:15:23 smithi154 podman[69846]: 2024-04-26 22:15:23.67011556 +0000 UTC m=+0.517723092 container remove a7dc7f9693b89eb48cc1795a52785d133e87360ad90023a029943307c2bd3fb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T22:15:23.688 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T22:15:23.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:23.724 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T22:15:23.725 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T22:15:23.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.a 2024-04-26T22:15:24.073 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:23 smithi012 systemd[1]: Stopping Ceph mds.a for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:24.573 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a[69696]: 2024-04-26T22:15:24.300+0000 7fd28ee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:24.573 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:24 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a[69696]: 2024-04-26T22:15:24.300+0000 7fd28ee00700 -1 mds.a *** got signal Terminated *** 2024-04-26T22:15:27.073 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:26 smithi012 podman[89465]: 2024-04-26 22:15:26.757390493 +0000 UTC m=+2.582965091 container died 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T22:15:27.429 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:27 smithi012 podman[89465]: 2024-04-26 22:15:27.203392561 +0000 UTC m=+3.028967152 container cleanup 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T22:15:27.429 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:27 smithi012 bash[89465]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a 2024-04-26T22:15:27.823 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 26 22:15:27 smithi012 podman[89490]: 2024-04-26 22:15:27.429190853 +0000 UTC m=+0.652421654 container remove 7993b81c1ce6a2ed4612a18f19d13af88c78137dc4cd96df1ee7b3619cf76705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T22:15:27.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T22:15:28.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:28.097 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T22:15:28.097 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T22:15:28.097 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.d 2024-04-26T22:15:28.573 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:28 smithi012 systemd[1]: Stopping Ceph mds.d for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:29.323 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d[68700]: 2024-04-26T22:15:28.881+0000 7f063fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:29.323 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:28 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d[68700]: 2024-04-26T22:15:28.881+0000 7f063fa00700 -1 mds.d *** got signal Terminated *** 2024-04-26T22:15:31.980 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:31 smithi012 podman[89594]: 2024-04-26 22:15:31.668952336 +0000 UTC m=+3.149077917 container died 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T22:15:32.323 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:32 smithi012 podman[89594]: 2024-04-26 22:15:32.106997781 +0000 UTC m=+3.587123408 container cleanup 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:15:32.323 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:32 smithi012 bash[89594]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d 2024-04-26T22:15:32.823 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 26 22:15:32 smithi012 podman[89608]: 2024-04-26 22:15:32.374538219 +0000 UTC m=+0.682924831 container remove 738205e92810820e2bed9c5276cf1f8e6f8e0d30280604ba634bd62962da588e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T22:15:32.943 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T22:15:33.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:33.052 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T22:15:33.052 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T22:15:33.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.g 2024-04-26T22:15:33.573 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:33 smithi012 systemd[1]: Stopping Ceph mds.g for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:34.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: 2024-04-26T22:15:33.675+0000 7f22f1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:34.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:33 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g[69309]: 2024-04-26T22:15:33.675+0000 7f22f1000700 -1 mds.g *** got signal Terminated *** 2024-04-26T22:15:35.349 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:35 smithi012 podman[89715]: 2024-04-26 22:15:35.106847746 +0000 UTC m=+1.567953419 container died 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T22:15:35.670 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:35 smithi012 podman[89715]: 2024-04-26 22:15:35.460156404 +0000 UTC m=+1.921262091 container cleanup 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T22:15:35.670 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:35 smithi012 bash[89715]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g 2024-04-26T22:15:36.073 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 26 22:15:35 smithi012 podman[89729]: 2024-04-26 22:15:35.669330044 +0000 UTC m=+0.544750745 container remove 38277650be1cfd3d1949cbe1e5f21ea211b4bb8ac9ef9d9b88bf677973b05554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:15:36.152 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T22:15:36.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:36.261 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T22:15:36.261 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T22:15:36.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.j 2024-04-26T22:15:36.823 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:36 smithi012 systemd[1]: Stopping Ceph mds.j for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:37.324 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j[68321]: 2024-04-26T22:15:36.987+0000 7f804c800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:37.324 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:36 smithi012 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j[68321]: 2024-04-26T22:15:36.987+0000 7f804c800700 -1 mds.j *** got signal Terminated *** 2024-04-26T22:15:39.736 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:39 smithi012 podman[89835]: 2024-04-26 22:15:39.44041295 +0000 UTC m=+2.690192197 container died e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T22:15:40.007 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:39 smithi012 podman[89835]: 2024-04-26 22:15:39.847181877 +0000 UTC m=+3.096961130 container cleanup e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:15:40.007 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:39 smithi012 bash[89835]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j 2024-04-26T22:15:40.327 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 26 22:15:40 smithi012 podman[89861]: 2024-04-26 22:15:40.006443245 +0000 UTC m=+0.550498766 container remove e549a528f4866c16161dfa3ec04caaba3a2bd69d4969286bc9913fc765bbd0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:15:40.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T22:15:40.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:40.701 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T22:15:40.701 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T22:15:40.701 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.b 2024-04-26T22:15:40.982 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:40 smithi077 systemd[1]: Stopping Ceph mds.b for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:41.238 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: 2024-04-26T22:15:40.988+0000 7f614622e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:41.238 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:40 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b[62274]: 2024-04-26T22:15:40.988+0000 7f614622e700 -1 mds.b *** got signal Terminated *** 2024-04-26T22:15:43.397 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 podman[68816]: 2024-04-26 22:15:43.054785235 +0000 UTC m=+2.162012131 container died 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:15:43.672 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 podman[68816]: 2024-04-26 22:15:43.472193593 +0000 UTC m=+2.579420501 container cleanup 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T22:15:43.672 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 bash[68816]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b 2024-04-26T22:15:43.908 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T22:15:43.928 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 podman[68843]: 2024-04-26 22:15:43.673203848 +0000 UTC m=+0.611343576 container remove 26be940ac9ec6102806528ab1e50e171ff5b3b97898c8f1d2dd981315a5ff10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T22:15:43.928 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-26T22:15:43.929 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 systemd[1]: Stopped Ceph mds.b for d6d929b6-0416-11ef-bc93-c7b262605968. 2024-04-26T22:15:43.929 INFO:journalctl@ceph.mds.b.smithi077.stdout:Apr 26 22:15:43 smithi077 systemd[1]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.b.service: Consumed 3.640s CPU time. 2024-04-26T22:15:43.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:43.942 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T22:15:43.942 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T22:15:43.942 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.e 2024-04-26T22:15:44.232 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:44 smithi077 systemd[1]: Stopping Ceph mds.e for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:44.732 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e[61653]: 2024-04-26T22:15:44.259+0000 7f2303e53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:44.732 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:44 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e[61653]: 2024-04-26T22:15:44.259+0000 7f2303e53700 -1 mds.e *** got signal Terminated *** 2024-04-26T22:15:46.942 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:46 smithi077 podman[68950]: 2024-04-26 22:15:46.686594475 +0000 UTC m=+2.524582968 container died fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T22:15:47.232 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:47 smithi077 podman[68950]: 2024-04-26 22:15:47.042376982 +0000 UTC m=+2.880365473 container cleanup fcec7ef69cb350d2eee197b550c9e6bc7fdf682541394e403412edba381c2d48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T22:15:47.232 INFO:journalctl@ceph.mds.e.smithi077.stdout:Apr 26 22:15:47 smithi077 bash[68950]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-e 2024-04-26T22:15:47.468 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T22:15:47.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:47.501 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T22:15:47.501 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T22:15:47.501 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.h 2024-04-26T22:15:47.732 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:47 smithi077 systemd[1]: Stopping Ceph mds.h for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:48.232 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h[61965]: 2024-04-26T22:15:47.829+0000 7f5901409700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:48.232 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:47 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h[61965]: 2024-04-26T22:15:47.829+0000 7f5901409700 -1 mds.h *** got signal Terminated *** 2024-04-26T22:15:51.982 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:51 smithi077 podman[69072]: 2024-04-26 22:15:51.651739778 +0000 UTC m=+3.935891456 container died 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:15:52.480 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:52 smithi077 podman[69072]: 2024-04-26 22:15:52.079279741 +0000 UTC m=+4.363431430 container cleanup 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T22:15:52.480 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:52 smithi077 bash[69072]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h 2024-04-26T22:15:52.481 INFO:journalctl@ceph.mds.h.smithi077.stdout:Apr 26 22:15:52 smithi077 podman[69086]: 2024-04-26 22:15:52.255975604 +0000 UTC m=+0.607391369 container remove 04a979fe9e634bfa48d1066800cb11e47c2ecd76fd8437813402bf8931b4d47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T22:15:52.497 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T22:15:52.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:52.530 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T22:15:52.530 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T22:15:52.530 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.k 2024-04-26T22:15:52.841 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:52 smithi077 systemd[1]: Stopping Ceph mds.k for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:53.232 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k[61337]: 2024-04-26T22:15:52.841+0000 7f3790375700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:53.232 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:52 smithi077 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k[61337]: 2024-04-26T22:15:52.841+0000 7f3790375700 -1 mds.k *** got signal Terminated *** 2024-04-26T22:15:55.732 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:55 smithi077 podman[69195]: 2024-04-26 22:15:55.445433025 +0000 UTC m=+2.701433394 container died 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:15:56.042 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:55 smithi077 podman[69195]: 2024-04-26 22:15:55.8250629 +0000 UTC m=+3.081063276 container cleanup 339bd94210aaab3f0ea4d3b75093fe9c2c62235ad6a7e19217c1cd6080b178d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T22:15:56.042 INFO:journalctl@ceph.mds.k.smithi077.stdout:Apr 26 22:15:55 smithi077 bash[69195]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-k 2024-04-26T22:15:56.174 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T22:15:56.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:15:56.205 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T22:15:56.205 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T22:15:56.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.c 2024-04-26T22:15:56.501 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:15:56 smithi154 systemd[1]: Stopping Ceph mds.c for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:15:56.764 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:15:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c[63132]: 2024-04-26T22:15:56.501+0000 7f34b2b00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:15:56.764 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:15:56 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c[63132]: 2024-04-26T22:15:56.501+0000 7f34b2b00700 -1 mds.c *** got signal Terminated *** 2024-04-26T22:16:00.885 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:16:00 smithi154 podman[69924]: 2024-04-26 22:16:00.515806324 +0000 UTC m=+4.118155275 container died 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:16:01.140 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:16:00 smithi154 podman[69924]: 2024-04-26 22:16:00.968131565 +0000 UTC m=+4.570480523 container cleanup 0b215deba1c0aab96955a8805d63e4a87d0ccbe3771c2fe6f2f114b8a16e7c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T22:16:01.141 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 26 22:16:00 smithi154 bash[69924]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-c 2024-04-26T22:16:01.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T22:16:01.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:16:01.419 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T22:16:01.419 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T22:16:01.419 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.f 2024-04-26T22:16:01.640 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:01 smithi154 systemd[1]: Stopping Ceph mds.f for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:16:02.141 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f[62829]: 2024-04-26T22:16:01.739+0000 7f006772d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:16:02.141 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:01 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f[62829]: 2024-04-26T22:16:01.739+0000 7f006772d700 -1 mds.f *** got signal Terminated *** 2024-04-26T22:16:06.006 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:05 smithi154 podman[70056]: 2024-04-26 22:16:05.75173804 +0000 UTC m=+4.114406928 container died 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T22:16:06.341 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:06 smithi154 podman[70056]: 2024-04-26 22:16:06.102145759 +0000 UTC m=+4.464814679 container cleanup 295891c0a8b1f8f2e69f1b57559883dade7c91278b0f4bde70343e706cf72ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T22:16:06.341 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 26 22:16:06 smithi154 bash[70056]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-f 2024-04-26T22:16:06.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T22:16:06.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:16:06.505 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T22:16:06.505 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T22:16:06.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.i 2024-04-26T22:16:06.640 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:06 smithi154 systemd[1]: Stopping Ceph mds.i for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:16:07.140 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i[63436]: 2024-04-26T22:16:06.842+0000 7f465425f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:16:07.141 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:06 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i[63436]: 2024-04-26T22:16:06.842+0000 7f465425f700 -1 mds.i *** got signal Terminated *** 2024-04-26T22:16:10.890 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:10 smithi154 podman[70176]: 2024-04-26 22:16:10.569938619 +0000 UTC m=+3.849141683 container died 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T22:16:11.390 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:11 smithi154 podman[70176]: 2024-04-26 22:16:11.14300724 +0000 UTC m=+4.422210304 container cleanup 96dfd8e6f8ecabdf197d994616f185738dea62b4178e4fd4c953bb5d14795fed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T22:16:11.391 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 26 22:16:11 smithi154 bash[70176]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-i 2024-04-26T22:16:11.612 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T22:16:11.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:16:11.645 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T22:16:11.645 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T22:16:11.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.l 2024-04-26T22:16:11.890 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:11 smithi154 systemd[1]: Stopping Ceph mds.l for d6d929b6-0416-11ef-bc93-c7b262605968... 2024-04-26T22:16:12.390 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l[62516]: 2024-04-26T22:16:11.963+0000 7f866a7d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:16:12.390 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:11 smithi154 ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l[62516]: 2024-04-26T22:16:11.963+0000 7f866a7d2700 -1 mds.l *** got signal Terminated *** 2024-04-26T22:16:16.256 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:15 smithi154 podman[70297]: 2024-04-26 22:16:15.973075387 +0000 UTC m=+4.113868954 container died 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T22:16:16.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:16 smithi154 podman[70297]: 2024-04-26 22:16:16.32314966 +0000 UTC m=+4.463943232 container cleanup 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T22:16:16.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:16 smithi154 bash[70297]: ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l 2024-04-26T22:16:16.641 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 26 22:16:16 smithi154 podman[70311]: 2024-04-26 22:16:16.482248673 +0000 UTC m=+0.503723172 container remove 63fb37f882067f6e1baa2f0855093af3fab91a77c3a10ef4ab236a4f0415b970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-d6d929b6-0416-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T22:16:16.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6d929b6-0416-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T22:16:16.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:16:16.723 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T22:16:16.723 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:16:17.055 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:22.149 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:16:22.337 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:24.687 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:16:24.877 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:27.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:16:27.336 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:16:27.368 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:16:27.393 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T22:16:27.395 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi012/crash 2024-04-26T22:16:27.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:16:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:16:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:16:27.499 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi077/crash 2024-04-26T22:16:27.500 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:16:27.548 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:16:27.548 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:16:27.550 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi154/crash 2024-04-26T22:16:27.551 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:16:27.585 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:16:27.585 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:16:27.586 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T22:16:27.587 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T22:16:27.692 INFO:tasks.cephadm:Compressing logs... 2024-04-26T22:16:27.692 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:16:27.696 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:16:27.699 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:: No such file or directory 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T22:16:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.6% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:16:27.739 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.8.log: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:16:27.740 INFO:teuthology.orchestra.run.smithi077.stderr:: No such file or directory 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.6% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:gzip/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.h.log: -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T22:16:27.741 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T22:16:27.742 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:16:27.743 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:16:27.743 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T22:16:27.745 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:16:27.745 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:16:27.746 INFO:teuthology.orchestra.run.smithi077.stderr: 91.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:16:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:16:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:16:27.757 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:16:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T22:16:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T22:16:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.g.log: 87.6% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:16:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T22:16:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T22:16:27.762 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:16:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T22:16:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T22:16:27.766 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T22:16:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T22:16:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:16:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T22:16:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:16:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:16:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T22:16:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T22:16:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T22:16:27.779 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:16:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-26T22:16:27.787 INFO:teuthology.orchestra.run.smithi012.stderr: 94.5% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T22:16:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-26T22:16:27.801 INFO:teuthology.orchestra.run.smithi012.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:16:27.807 INFO:teuthology.orchestra.run.smithi012.stderr: 90.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:16:27.877 INFO:teuthology.orchestra.run.smithi012.stderr: 93.6% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T22:16:28.078 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T22:16:28.160 INFO:teuthology.orchestra.run.smithi077.stderr: 92.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T22:16:28.200 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T22:16:28.288 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T22:16:28.767 INFO:teuthology.orchestra.run.smithi012.stderr: 90.4% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T22:16:31.775 INFO:teuthology.orchestra.run.smithi154.stderr: 92.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T22:16:33.191 INFO:teuthology.orchestra.run.smithi154.stderr: 93.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T22:16:33.874 INFO:teuthology.orchestra.run.smithi077.stderr: 93.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T22:16:34.333 INFO:teuthology.orchestra.run.smithi077.stderr: 93.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T22:16:34.586 INFO:teuthology.orchestra.run.smithi154.stderr: 92.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T22:16:34.822 INFO:teuthology.orchestra.run.smithi012.stderr: 92.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T22:16:35.420 INFO:teuthology.orchestra.run.smithi077.stderr: 93.0% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T22:16:35.493 INFO:teuthology.orchestra.run.smithi077.stderr: 92.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T22:16:35.494 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-04-26T22:16:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:real 0m7.782s 2024-04-26T22:16:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:user 0m28.268s 2024-04-26T22:16:35.495 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m0.959s 2024-04-26T22:16:35.832 INFO:teuthology.orchestra.run.smithi012.stderr: 92.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T22:16:35.931 INFO:teuthology.orchestra.run.smithi012.stderr: 92.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T22:16:37.601 INFO:teuthology.orchestra.run.smithi154.stderr: 92.9% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T22:16:37.603 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-04-26T22:16:37.603 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m9.891s 2024-04-26T22:16:37.604 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m25.847s 2024-04-26T22:16:37.604 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.895s 2024-04-26T22:16:37.952 INFO:teuthology.orchestra.run.smithi012.stderr: 92.8% -- replaced with /var/log/ceph/d6d929b6-0416-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T22:16:37.956 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-26T22:16:37.956 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m10.227s 2024-04-26T22:16:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m32.336s 2024-04-26T22:16:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m2.686s 2024-04-26T22:16:37.958 INFO:tasks.cephadm:Archiving logs... 2024-04-26T22:16:37.958 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi012/log 2024-04-26T22:16:37.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:16:43.096 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi077/log 2024-04-26T22:16:43.097 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:16:47.699 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi154/log 2024-04-26T22:16:47.699 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:16:51.782 INFO:tasks.cephadm:Removing cluster... 2024-04-26T22:16:51.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force 2024-04-26T22:16:52.117 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:52.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force 2024-04-26T22:16:53.059 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:53.404 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6d929b6-0416-11ef-bc93-c7b262605968 --force 2024-04-26T22:16:53.597 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: d6d929b6-0416-11ef-bc93-c7b262605968 2024-04-26T22:16:53.888 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T22:16:53.888 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:16:53.941 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:16:53.956 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:16:53.971 INFO:tasks.cephadm:Teardown complete 2024-04-26T22:16:53.972 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T22:16:53.985 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T22:16:53.985 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:16:53.992 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:16:54.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:16:54.139 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T22:16:54.150 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T22:16:54.150 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:16:54.157 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:16:54.160 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:16:54.171 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:16:54.173 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +1500us[+1499us] +/- 47ms 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 +182us[ +182us] +/- 63ms 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -2565us[-2565us] +/- 74ms 2024-04-26T22:16:54.175 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:16:54.177 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:16:54.178 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-26T22:16:54.178 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +1521us[+1555us] +/- 47ms 2024-04-26T22:16:54.178 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +160us[ +195us] +/- 63ms 2024-04-26T22:16:54.178 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -2571us[-2537us] +/- 74ms 2024-04-26T22:16:54.178 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:16:54.202 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +1515us[+1522us] +/- 47ms 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 +228us[ +228us] +/- 63ms 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -2554us[-2554us] +/- 74ms 2024-04-26T22:16:54.210 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:16:54.211 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T22:16:54.222 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T22:16:54.223 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T22:16:54.234 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:16:54.436 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:16:54.500 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:16:54.565 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:16:54.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:54.662 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:16:54.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:54.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:16:54.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:54.717 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T22:16:54.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T22:16:54.739 INFO:teuthology.task.internal:Duration was 2087.461207 seconds 2024-04-26T22:16:54.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T22:16:54.750 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T22:16:54.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:16:54.759 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:16:54.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:16:54.795 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:16:54.798 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:16:54.904 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:16:55.270 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T22:16:55.270 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T22:16:55.270 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:16:55.337 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T22:16:55.337 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:16:55.359 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-04-26T22:16:55.360 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:16:55.387 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T22:16:55.387 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:16:55.391 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:16:55.402 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:16:55.461 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T22:16:55.462 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:16:55.466 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:16:55.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:16:55.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T22:16:55.980 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T22:16:55.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:16:56.068 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:16:56.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:16:56.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T22:16:56.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.136 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.159 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-04-26T22:16:56.179 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-04-26T22:16:56.235 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-04-26T22:16:56.281 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:56.331 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:56.345 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:16:56.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:16:56.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T22:16:56.372 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T22:16:56.372 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi012 2024-04-26T22:16:56.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:16:56.481 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi077 2024-04-26T22:16:56.481 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:16:56.516 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675232/remote/smithi154 2024-04-26T22:16:56.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:16:56.552 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T22:16:56.552 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:16:56.560 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:16:56.562 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:16:56.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T22:16:56.625 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T22:16:56.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T22:16:56.640 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T22:16:56.641 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:16:56.655 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:16:56.658 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:16:56.670 INFO:teuthology.orchestra.run.smithi077.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:16 /home/ubuntu/cephtest 2024-04-26T22:16:56.674 INFO:teuthology.orchestra.run.smithi154.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:16 /home/ubuntu/cephtest 2024-04-26T22:16:56.705 INFO:teuthology.orchestra.run.smithi012.stdout: 393243 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:16 /home/ubuntu/cephtest 2024-04-26T22:16:56.710 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T22:16:56.765 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T22:16:56.882 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} duration: 2087.4612073898315 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T22:16:56.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:16:56.970 INFO:teuthology.run:pass